-- Logs begin at Mon 2022-09-12 13:34:01 UTC, end at Mon 2022-09-12 14:34:07 UTC. -- Sep 12 13:58:45.070045 dv-cinder-820027-25-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 12 13:58:45.079483 dv-cinder-820027-25-iscsi devstack@keystone.service[81583]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 12 13:58:45.081496 dv-cinder-820027-25-iscsi devstack@keystone.service[81583]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Sep 12 13:58:45 2022] *** Sep 12 13:58:45.081496 dv-cinder-820027-25-iscsi devstack@keystone.service[81583]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 12 13:58:45.081496 dv-cinder-820027-25-iscsi devstack@keystone.service[81583]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 12 13:58:45.081496 dv-cinder-820027-25-iscsi devstack@keystone.service[81583]: nodename: dv-cinder-820027-25-iscsi Sep 12 13:58:45.081496 dv-cinder-820027-25-iscsi devstack@keystone.service[81583]: machine: x86_64 Sep 12 13:58:45.081496 dv-cinder-820027-25-iscsi devstack@keystone.service[81583]: clock source: unix Sep 12 13:58:45.081496 dv-cinder-820027-25-iscsi devstack@keystone.service[81583]: pcre jit disabled Sep 12 13:58:45.081496 dv-cinder-820027-25-iscsi devstack@keystone.service[81583]: detected number of CPU cores: 4 Sep 12 13:58:45.081496 dv-cinder-820027-25-iscsi devstack@keystone.service[81583]: current working directory: / Sep 12 13:58:45.081496 dv-cinder-820027-25-iscsi devstack@keystone.service[81583]: detected binary path: /usr/bin/uwsgi-core Sep 12 13:58:45.081496 dv-cinder-820027-25-iscsi devstack@keystone.service[81583]: your processes number limit is 47680 Sep 12 13:58:45.082050 dv-cinder-820027-25-iscsi devstack@keystone.service[81583]: your memory page size is 4096 bytes Sep 12 13:58:45.082050 dv-cinder-820027-25-iscsi devstack@keystone.service[81583]: detected max file descriptor number: 2048 Sep 12 13:58:45.082050 dv-cinder-820027-25-iscsi devstack@keystone.service[81583]: lock engine: pthread robust mutexes Sep 12 13:58:45.082050 dv-cinder-820027-25-iscsi devstack@keystone.service[81583]: thunder lock: enabled Sep 12 13:58:45.082050 dv-cinder-820027-25-iscsi devstack@keystone.service[81583]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 12 13:58:45.082050 dv-cinder-820027-25-iscsi devstack@keystone.service[81583]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 12 13:58:45.120473 dv-cinder-820027-25-iscsi devstack@keystone.service[81583]: Python main interpreter initialized at 0x5588a66fd9c0 Sep 12 13:58:45.120473 dv-cinder-820027-25-iscsi devstack@keystone.service[81583]: python threads support enabled Sep 12 13:58:45.120473 dv-cinder-820027-25-iscsi devstack@keystone.service[81583]: your server socket listen backlog is limited to 100 connections Sep 12 13:58:45.120473 dv-cinder-820027-25-iscsi devstack@keystone.service[81583]: your mercy for graceful operations on workers is 90 seconds Sep 12 13:58:45.120790 dv-cinder-820027-25-iscsi devstack@keystone.service[81583]: mapped 403077 bytes (393 KB) for 2 cores Sep 12 13:58:45.120869 dv-cinder-820027-25-iscsi devstack@keystone.service[81583]: *** Operational MODE: preforking *** Sep 12 13:58:45.121598 dv-cinder-820027-25-iscsi devstack@keystone.service[81583]: *** uWSGI is running in multiple interpreter mode *** Sep 12 13:58:45.121598 dv-cinder-820027-25-iscsi devstack@keystone.service[81583]: spawned uWSGI master process (pid: 81583) Sep 12 13:58:45.121785 dv-cinder-820027-25-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 12 13:58:45.123234 dv-cinder-820027-25-iscsi devstack@keystone.service[81583]: spawned uWSGI worker 1 (pid: 81596, cores: 1) Sep 12 13:58:45.124692 dv-cinder-820027-25-iscsi devstack@keystone.service[81583]: spawned uWSGI worker 2 (pid: 81597, cores: 1) Sep 12 13:58:45.124692 dv-cinder-820027-25-iscsi devstack@keystone.service[81583]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 12 13:58:49.409139 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 12 13:58:49.409139 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: from cryptography.utils import int_from_bytes Sep 12 13:58:49.409936 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 12 13:58:49.409936 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: from cryptography.utils import int_from_bytes Sep 12 13:58:49.410227 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 12 13:58:49.410227 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: from cryptography.utils import int_from_bytes Sep 12 13:58:49.410679 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 12 13:58:49.410679 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: from cryptography.utils import int_from_bytes Sep 12 13:58:49.459726 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 12 13:58:49.459891 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 12 13:58:49.459967 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 12 13:58:49.460048 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 12 13:58:49.460125 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] command line args: [] {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 12 13:58:49.460237 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] command line args: [] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 12 13:58:49.460356 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 12 13:58:49.460509 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 12 13:58:49.460593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ================================================================================ {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 12 13:58:49.460693 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ================================================================================ {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 12 13:58:49.460777 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] admin_token = **** {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.460949 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] admin_token = **** {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.461038 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] config_dir = [] {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.461187 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.461270 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] config_dir = [] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.461373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.461454 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] config_source = [] {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.461553 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] debug = True {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.461640 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] config_source = [] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.461732 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] debug = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.461813 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.461993 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.462149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.462250 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.462344 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.462437 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.462536 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.462635 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.462764 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.462902 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.463015 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] list_limit = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.463162 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] list_limit = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.463300 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] log_config_append = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.463401 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] log_config_append = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.463505 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.463600 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.463706 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] log_dir = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.463800 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] log_dir = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.463899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] log_file = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.464007 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] log_file = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.464092 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.464215 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.464296 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.464371 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.464487 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.464560 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.464680 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.464792 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.465402 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.465478 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.465593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.465678 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.465750 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.465864 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.465981 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.466124 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.466208 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.466286 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.466440 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.466525 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.466674 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.466756 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.466931 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.467035 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.467182 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.467325 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.467429 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.467505 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.467704 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.467792 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.468191 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.468314 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.468397 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] publish_errors = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.468468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] publish_errors = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.468592 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.468672 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.468807 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.468885 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.469017 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.469102 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.469225 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.469318 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.469459 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.469536 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.469694 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] standard_threads = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.469779 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] standard_threads = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.469925 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.470008 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.470167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.470258 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.470330 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.470403 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.470528 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] use_journal = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.470683 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] use_journal = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.470683 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] use_json = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.470815 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] use_json = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.470901 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] use_stderr = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.470971 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] use_stderr = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.471039 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] use_syslog = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.471122 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] use_syslog = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.471243 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.471322 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 13:58:49.471499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.471575 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.471737 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.471821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.471994 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.472068 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.472231 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.472306 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.472479 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.472571 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.472715 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.472791 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.472963 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.473043 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.473197 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.473281 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.473443 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.473527 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.473687 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.473770 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.473934 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.474018 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.474167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.474243 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.474412 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.474502 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.474646 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.474777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.474913 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.475000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.475161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.475236 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.475396 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.475479 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.475650 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.475728 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.475902 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.475977 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.476144 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.476222 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.476377 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.476452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.476628 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.476703 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.476888 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.476965 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.477136 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.477211 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.477383 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.477465 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.477612 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.477688 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.477841 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.477923 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.478069 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.478144 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.478302 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.478385 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.478546 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.478622 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.478791 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.478909 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.479046 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.479123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.479276 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.479352 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.479519 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.479593 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.479766 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.479842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.479999 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.480077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.480247 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.480323 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.480509 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.480589 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.480744 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.480818 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.480977 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.481059 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.481207 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.481283 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.481434 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.481508 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.481663 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.481746 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.481897 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.481975 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.482114 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.482192 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.482359 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.482434 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.482590 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.482672 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.482822 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.482928 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.483076 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.483193 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.483311 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.483412 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.483544 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.483644 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.483886 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.484022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.484022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.484198 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.484312 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.484386 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.484499 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.484580 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.484695 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.484768 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.484908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.485024 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.485143 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.485229 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.485371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.485446 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.485620 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.485707 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.485856 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.485940 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.486089 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.486169 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.486320 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.486419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.486574 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.486660 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.486809 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.486952 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.487063 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.487139 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.487299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.487374 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.487533 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.487620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.487799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.487878 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.488035 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.488113 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.488281 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.488409 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.488524 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.488601 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.488758 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.488835 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.488991 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.489076 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.489225 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.489301 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.489471 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.489548 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.489708 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.489792 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.489943 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.490019 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.490184 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.490259 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.490418 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.490503 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.490662 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.490738 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.490920 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.491003 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.491159 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.491252 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.491395 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.491474 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.491629 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.491704 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.491862 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.491946 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.492120 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.492197 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.492359 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.492441 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.492603 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.492688 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.492840 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.492916 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.493073 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.493148 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.493314 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.493461 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.493536 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.493605 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.493769 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.493847 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.494002 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.494091 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.494228 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.494311 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.494455 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.494531 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.494682 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.494765 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.494938 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.495017 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.495167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.495245 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.495404 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.495480 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.495637 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.495714 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.495883 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.495976 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.496142 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.496219 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.496395 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.496472 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.496651 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.496733 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.496891 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.496992 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.497119 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] auth.external = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.497197 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] auth.external = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.497347 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.497425 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.497637 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.497714 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.497866 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.497942 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.498094 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] auth.password = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.498172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] auth.password = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.498322 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] auth.token = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.498398 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] auth.token = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.498560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.498636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.498795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.498892 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.499065 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.499215 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.499344 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.499470 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.499603 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.499730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.499855 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.499982 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.500107 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.500232 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.500364 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] credential.caching = True {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.500490 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] credential.caching = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.500615 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.500739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.500879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.501006 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.501141 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.501274 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.501399 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.501524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.501524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.501768 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.501905 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.502034 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.502166 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.502292 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.502416 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.502549 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.502687 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.502812 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.502985 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.503112 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.503238 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.503378 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.503502 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.503627 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.503752 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.503877 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.504001 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.504130 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.504258 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.504393 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.504516 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] federation.caching = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.504641 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] federation.caching = True {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.504772 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.504908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.505040 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.505166 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.505292 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.505417 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.505542 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.505674 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.505798 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.505924 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.506049 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.506182 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.506310 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.506442 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.506568 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.506692 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.506816 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.506954 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.507030 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.507117 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.507192 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.507270 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.507348 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] identity.caching = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.507420 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] identity.caching = True {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.507420 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.507669 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.507747 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.507818 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.507888 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.507958 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.508027 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.508105 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.508185 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.508257 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.508328 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.508397 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.508397 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.508531 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.508608 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.508680 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.508827 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.508911 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.509059 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.509144 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.509288 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.509366 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.509693 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.509775 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.509848 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.509927 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.510000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.510131 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.510210 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.510288 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.510462 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.510549 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.510708 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.510784 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.511000 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.511081 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.511241 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.511320 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.511481 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.512102 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.512222 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.512339 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.512475 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.512606 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.512744 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.512863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.512995 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.513099 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.513217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.513324 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.513427 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.513532 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.513636 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.513738 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.513853 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.513967 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.514075 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.514185 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.514325 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.514429 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.514556 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.514635 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.514801 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.514898 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.515073 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.515150 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.515303 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.515382 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.515537 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.515614 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.515766 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.515842 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.516001 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.516079 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.516236 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.516320 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.516473 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.516549 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.516711 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.516791 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.517063 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.517146 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.517315 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.517392 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.517564 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.517642 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.517797 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.517876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.518028 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.518105 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.518277 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.518355 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.518524 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.518601 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.518762 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.518889 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.519022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.519102 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.519259 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.519336 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.519478 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.user = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.519554 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.519718 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.519799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.519976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.520053 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.520207 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.520283 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.520452 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.520530 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.520697 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.520775 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.520943 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.521020 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.521177 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.521255 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.521405 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.521482 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.521639 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.521716 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.521873 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.521951 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.522108 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.522184 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.522335 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.522412 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.522580 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.522658 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.522824 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.522921 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.523092 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.523181 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.523337 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.523416 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.523580 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.523657 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.523809 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.523885 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.524053 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.524132 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.524288 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.524369 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.524522 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.524599 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.524756 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.524834 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.524989 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.525066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.525230 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.525306 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.525475 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.525554 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.525711 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.525787 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.525963 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.526039 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.526193 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.526272 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.526436 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.526512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.526669 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.526754 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.526928 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.527009 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.527166 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.527829 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.527999 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.528093 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.528244 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.528323 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.528476 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.528553 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.528711 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.528795 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.528950 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] resource.caching = True {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.529026 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] resource.caching = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.529197 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.529274 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.529442 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.529527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.529677 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.529754 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.529923 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.530000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.530171 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.530261 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.530405 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.530482 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.530650 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.530726 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.530911 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.531065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.531141 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.531211 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.531377 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] role.caching = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.531456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] role.caching = True {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.531623 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] role.driver = sql {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.531718 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] role.driver = sql {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.531854 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.531931 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.532102 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.532178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.532349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.532435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.532594 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.532671 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.532840 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.532916 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.533085 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.533170 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.533329 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.533406 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.533571 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.533648 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.533815 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.533901 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.534045 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.534122 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.534289 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.534372 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.534536 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.534622 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.534781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.534877 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.535054 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.535132 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.535453 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.535532 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.535687 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.535764 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.535936 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.536013 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.536187 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.536267 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.536434 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.536510 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.536682 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.536758 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.536916 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.536994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.537169 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.537287 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.537403 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.537477 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.537649 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.537736 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.537880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.537958 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.538110 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.538186 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.538339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.538417 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.538589 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.538666 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.538881 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.538962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.539113 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.539192 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.539350 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.539427 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.539584 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.539660 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.539814 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.539892 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.540047 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] token.caching = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.540130 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] token.caching = True {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.540281 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.540357 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.540527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.540606 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.540762 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.540839 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.541017 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.541093 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.541261 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.541340 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.541497 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.541574 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.541739 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.541816 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.541982 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.542060 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.542227 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.542304 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.542472 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.542568 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.542696 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.542780 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.542959 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.543037 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.543211 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.543290 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.543482 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.543561 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.543711 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.543798 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.543952 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.544039 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.544212 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.544327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.544469 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.connection = **** {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.544555 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] database.connection = **** {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.544710 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.544787 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.544953 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.545031 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.545188 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.545273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.545427 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.545504 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.545658 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.545735 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.545892 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.545970 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.546124 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.546201 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.546358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.546434 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.546599 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.546677 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.546832 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.546966 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.547098 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.547176 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.547332 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.547421 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.547722 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.547794 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.547865 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.547936 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.548015 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.548088 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.548247 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.548325 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.548483 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.548560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.548715 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.548793 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 13:58:49.548885 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 12 13:58:49.548971 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 12 13:58:49.579356 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81596) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.579495 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.579703 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.579786 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81596) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.579933 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.580016 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81596) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.588395 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.591162 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81596) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.618871 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.619100 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.619509 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81596) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.619746 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81596) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.673076 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.673562 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81596) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.722763 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.723022 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.723816 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81596) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.724060 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81596) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.878206 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81596) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.878396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.878484 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81596) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.878629 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.878781 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81596) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.878892 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81596) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.878976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.879081 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81596) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.879167 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.879296 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.879382 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81596) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.879518 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.904181 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.904406 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.904639 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.904871 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.904994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81596) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.905079 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.905231 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81596) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.905313 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.905467 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81596) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.905558 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.905695 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81596) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.905919 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81596) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.906140 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81596) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.906358 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81596) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.918614 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81596) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.918888 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.932945 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.933124 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.933355 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.934231 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.934375 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81596) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.934461 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.934613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81596) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.934695 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.934859 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81596) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.934935 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.935092 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81596) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.935313 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81596) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.935524 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81596) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.935743 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81596) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 13:58:49.951728 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.951942 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 12 13:58:49.952089 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.952217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 12 13:58:49.952354 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.952480 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 12 13:58:49.952604 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.952729 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 12 13:58:49.952852 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.952977 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.953109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.953241 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.953366 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.953490 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.953623 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.953749 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.962998 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.963233 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.963365 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.963490 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.963614 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.963748 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.963879 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.964034 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.970743 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 13:58:49.971027 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81596) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 13:58:49.973634 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 13:58:49.973902 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81596) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 13:58:49.973985 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.974192 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.974273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.974476 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.974565 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.974763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.974861 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.975056 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.975137 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.975333 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.975416 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.975604 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.975682 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.975875 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.975963 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.976146 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.976224 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.976417 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.976496 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:49.976688 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.000097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.000222 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.001569 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 13:58:50.001685 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81596) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 13:58:50.001857 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.001974 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.006484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 13:58:50.006626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81596) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 13:58:50.006774 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.006939 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.007087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.007230 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.013401 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 13:58:50.013550 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81596) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 13:58:50.013692 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.013845 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.019299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 13:58:50.019476 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81596) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 13:58:50.019598 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.019774 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.019891 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.020065 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.020183 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.020355 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.020488 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.020638 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.020728 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.020918 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.020998 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.021193 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.021272 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.021466 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.036688 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.036838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.036971 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.037122 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.040843 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 13:58:50.040978 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81596) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 13:58:50.044596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.044751 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.044897 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.045053 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.049073 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 13:58:50.049251 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81596) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 13:58:50.052227 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 13:58:50.052371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81596) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 13:58:50.055448 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.055585 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.055756 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.055892 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.056049 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.056187 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.056337 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.056480 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.056626 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.056767 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.056916 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.057072 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.073352 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 13:58:50.073510 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81596) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 13:58:50.073642 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.073802 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.073915 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.074120 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.074318 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.074512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.081187 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.081369 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.082999 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.084527 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.084820 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.087766 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81596) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 13:58:50.088057 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.088345 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.088633 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.089040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.090925 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.092481 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.092774 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.095725 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 13:58:50.096016 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.096304 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.096595 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.096995 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.100154 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81596) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 13:58:50.100443 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.100726 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.101011 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.101397 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.101698 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.101985 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.108640 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 13:58:50.108931 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.109224 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.109508 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.109893 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.110190 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.110473 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.117827 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81596) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 13:58:50.120770 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81596) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 13:58:50.124222 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.125662 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.126288 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 13:58:50.127171 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81596) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 13:58:50.127458 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.127855 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.129328 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 13:58:50.132240 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.133682 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.134193 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.135192 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 13:58:50.135481 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.135664 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81596) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 13:58:50.135884 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.138564 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.138991 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.139285 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.139577 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.142224 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.143721 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 13:58:50.146578 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.147004 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.147299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.147379 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81596) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 13:58:50.147587 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.147669 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.147881 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.154739 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81596) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 13:58:50.155049 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.155334 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.155415 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 13:58:50.155614 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.155696 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.155891 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.155972 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.156168 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.156445 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.156733 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.157016 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.157305 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.157581 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.157846 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.158114 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.158446 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81596) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.162741 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 13:58:50.163056 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.163344 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.163629 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.163911 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.164187 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.164463 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.164751 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.165035 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.165317 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.165613 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.165879 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.166149 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.166429 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 13:58:50.193266 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5588a66fd9c0 pid: 81596 (default app) Sep 12 13:58:50.196503 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af46d09-77c4-443f-9e41-0f7ea8e24160 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:58:50.197049 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af46d09-77c4-443f-9e41-0f7ea8e24160 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:58:50.197327 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af46d09-77c4-443f-9e41-0f7ea8e24160 None None] PATH_INFO: `/v3` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:58:50.201354 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5588a66fd9c0 pid: 81597 (default app) Sep 12 13:58:50.201518 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1/1] 10.222.0.42 () {50 vars in 791 bytes} [Mon Sep 12 13:58:50 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:58:56.933862 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd11ab3-51c9-42aa-b754-087892437208 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:58:56.934404 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd11ab3-51c9-42aa-b754-087892437208 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:58:56.934404 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd11ab3-51c9-42aa-b754-087892437208 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:58:56.938631 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1/2] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:58:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:58:56.944288 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4504e5a-869f-4987-9b96-ba8f07985c9d None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:58:56.944817 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4504e5a-869f-4987-9b96-ba8f07985c9d None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:58:56.944817 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4504e5a-869f-4987-9b96-ba8f07985c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:58:57.015740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: INFO dbcounter [None req-b4504e5a-869f-4987-9b96-ba8f07985c9d None None] Registered counter for database keystone Sep 12 13:58:57.034651 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG oslo_db.sqlalchemy.engines [None req-b4504e5a-869f-4987-9b96-ba8f07985c9d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81596) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 12 13:58:57.040892 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writer thread running {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 12 13:58:57.216332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG passlib.handlers.bcrypt [None req-b4504e5a-869f-4987-9b96-ba8f07985c9d None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81596) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 12 13:58:57.216646 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG passlib.handlers.bcrypt [None req-b4504e5a-869f-4987-9b96-ba8f07985c9d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81596) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 12 13:58:57.274332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-b4504e5a-869f-4987-9b96-ba8f07985c9d None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:58:57.275921 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-b4504e5a-869f-4987-9b96-ba8f07985c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:58:57.406460 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 2/3] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:58:56 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 12 13:58:57.868191 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd11ab3-51c9-42aa-b754-087892437208 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:58:57.937688 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: INFO dbcounter [None req-fdd11ab3-51c9-42aa-b754-087892437208 None None] Registered counter for database keystone Sep 12 13:58:57.958806 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG oslo_db.sqlalchemy.engines [None req-fdd11ab3-51c9-42aa-b754-087892437208 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81597) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 12 13:58:57.965450 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writer thread running {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 12 13:58:58.086631 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd11ab3-51c9-42aa-b754-087892437208 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:58:58.091382 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afdf059-356f-4a55-a5da-6cfbd81b792c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:58:58.092012 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afdf059-356f-4a55-a5da-6cfbd81b792c admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:58:58.092295 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afdf059-356f-4a55-a5da-6cfbd81b792c admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:58:58.092568 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afdf059-356f-4a55-a5da-6cfbd81b792c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:58:58.100668 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afdf059-356f-4a55-a5da-6cfbd81b792c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:58:58.151475 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7afdf059-356f-4a55-a5da-6cfbd81b792c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 12 13:58:58.151821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afdf059-356f-4a55-a5da-6cfbd81b792c admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:58:58.161503 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 2/4] 10.222.0.42 () {58 vars in 1324 bytes} [Mon Sep 12 13:58:57 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 13:58:58.167404 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4504e5a-869f-4987-9b96-ba8f07985c9d None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:58:58.184572 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4504e5a-869f-4987-9b96-ba8f07985c9d None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:58:58.188951 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef491a0-d42a-4b1a-aecf-db2d230f06f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:58:58.190049 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef491a0-d42a-4b1a-aecf-db2d230f06f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:58:58.190330 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef491a0-d42a-4b1a-aecf-db2d230f06f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:58:58.190604 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef491a0-d42a-4b1a-aecf-db2d230f06f5 admin admin] PATH_INFO: `/v3/projects/b159fcc4f05d45de88c679e235640813` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:58:58.192204 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef491a0-d42a-4b1a-aecf-db2d230f06f5 admin admin] RBAC: Authorizing `identity:get_project(project_id=b159fcc4f05d45de88c679e235640813)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:58:58.245285 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: WARNING keystone.common.rbac_enforcer.enforcer [None req-eef491a0-d42a-4b1a-aecf-db2d230f06f5 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 12 13:58:58.245629 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef491a0-d42a-4b1a-aecf-db2d230f06f5 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:58:58.250079 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 3/5] 10.222.0.42 () {56 vars in 1207 bytes} [Mon Sep 12 13:58:58 2022] GET /identity/v3/projects/b159fcc4f05d45de88c679e235640813 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:00.357705 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8631cc8a-e891-4433-9e1c-b05bbeeb5dfe None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:00.358235 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8631cc8a-e891-4433-9e1c-b05bbeeb5dfe None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:00.358235 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8631cc8a-e891-4433-9e1c-b05bbeeb5dfe None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:00.362365 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 3/6] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:00.367833 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969ce03-d88a-412b-8952-ecf589902569 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:00.368230 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969ce03-d88a-412b-8952-ecf589902569 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:00.368382 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969ce03-d88a-412b-8952-ecf589902569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:00.389203 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-f969ce03-d88a-412b-8952-ecf589902569 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:00.390710 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-f969ce03-d88a-412b-8952-ecf589902569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:00.398352 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 4/7] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:00 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 12 13:59:00.671418 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8631cc8a-e891-4433-9e1c-b05bbeeb5dfe None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:00.683859 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8631cc8a-e891-4433-9e1c-b05bbeeb5dfe None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:00.688162 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d6e13-9f40-406e-b3c6-41b7e8760101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:00.688758 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d6e13-9f40-406e-b3c6-41b7e8760101 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:00.689041 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d6e13-9f40-406e-b3c6-41b7e8760101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:00.689316 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d6e13-9f40-406e-b3c6-41b7e8760101 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:00.697207 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d6e13-9f40-406e-b3c6-41b7e8760101 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:00.699018 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d6e13-9f40-406e-b3c6-41b7e8760101 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:00.707577 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 4/8] 10.222.0.42 () {58 vars in 1324 bytes} [Mon Sep 12 13:59:00 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 13:59:00.712904 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f969ce03-d88a-412b-8952-ecf589902569 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:00.721951 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f969ce03-d88a-412b-8952-ecf589902569 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:00.726278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33bb2b8-3a23-48ee-a955-c7d711bc0469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:00.726929 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33bb2b8-3a23-48ee-a955-c7d711bc0469 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:00.727210 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33bb2b8-3a23-48ee-a955-c7d711bc0469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:00.727483 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33bb2b8-3a23-48ee-a955-c7d711bc0469 admin admin] PATH_INFO: `/v3/users/e46b19c26ed14ce08997d6b7fd3a2efb` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:00.728779 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33bb2b8-3a23-48ee-a955-c7d711bc0469 admin admin] RBAC: Authorizing `identity:get_user(user_id=e46b19c26ed14ce08997d6b7fd3a2efb)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:00.731207 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33bb2b8-3a23-48ee-a955-c7d711bc0469 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:00.732710 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 5/9] 10.222.0.42 () {56 vars in 1198 bytes} [Mon Sep 12 13:59:00 2022] GET /identity/v3/users/e46b19c26ed14ce08997d6b7fd3a2efb => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:02.908959 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8443f-fe39-45a9-8196-395770e581d2 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:02.909481 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8443f-fe39-45a9-8196-395770e581d2 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:02.909481 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8443f-fe39-45a9-8196-395770e581d2 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:02.910800 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 5/10] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 13:59:02.915869 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9893cdd-f1e1-4c0d-bfeb-1d02678e24d0 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:02.916299 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9893cdd-f1e1-4c0d-bfeb-1d02678e24d0 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:02.916425 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9893cdd-f1e1-4c0d-bfeb-1d02678e24d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:02.935219 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-b9893cdd-f1e1-4c0d-bfeb-1d02678e24d0 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:02.936383 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-b9893cdd-f1e1-4c0d-bfeb-1d02678e24d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:02.990632 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3983578-1b82-4ffd-874f-2f50b38e22ba None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:02.990946 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3983578-1b82-4ffd-874f-2f50b38e22ba None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:02.991232 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3983578-1b82-4ffd-874f-2f50b38e22ba None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:02.992554 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 6/11] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 13:59:02.998251 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1346dfb7-481e-4bae-8657-9d30dcf1c264 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:02.998606 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1346dfb7-481e-4bae-8657-9d30dcf1c264 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:02.998988 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1346dfb7-481e-4bae-8657-9d30dcf1c264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:03.016801 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 6/12] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:03.041187 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG passlib.handlers.bcrypt [None req-1346dfb7-481e-4bae-8657-9d30dcf1c264 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81597) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 12 13:59:03.041584 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG passlib.handlers.bcrypt [None req-1346dfb7-481e-4bae-8657-9d30dcf1c264 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81597) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 12 13:59:03.091436 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-1346dfb7-481e-4bae-8657-9d30dcf1c264 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:03.092953 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1346dfb7-481e-4bae-8657-9d30dcf1c264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:03.190201 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 7/13] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 13:59:03.284611 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9893cdd-f1e1-4c0d-bfeb-1d02678e24d0 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:03.308366 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:03.389142 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9893cdd-f1e1-4c0d-bfeb-1d02678e24d0 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:03.422694 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd38c912-827f-4704-88d7-9c235f880ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:03.423349 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38c912-827f-4704-88d7-9c235f880ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:03.423634 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38c912-827f-4704-88d7-9c235f880ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:03.423911 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38c912-827f-4704-88d7-9c235f880ba7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:03.434804 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd38c912-827f-4704-88d7-9c235f880ba7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:03.443720 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-dd38c912-827f-4704-88d7-9c235f880ba7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 13:59:03.443720 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:03.443720 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:03.443720 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:03.443720 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:03.443720 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:03.443720 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:03.443720 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:03.443720 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:03.443720 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:03.443720 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:03.443720 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:03.443720 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:03.443720 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:03.443720 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:03.443720 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:03.444591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:03.444591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:03.444591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:03.444591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:03.444591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:03.444591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:03.444591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:03.444591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:03.444591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:03.444591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:03.444591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:03.444591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:03.444591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:03.444591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:03.444591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:03.444591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:03.445659 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:03.445659 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:03.445659 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:03.445659 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:03.445659 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:03.445659 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:03.445659 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:03.445659 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:03.445659 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:03.445659 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:03.445659 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 13:59:03.445659 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 13:59:03.445659 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 7/14] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 12 13:59:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 13:59:03.451492 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1346dfb7-481e-4bae-8657-9d30dcf1c264 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:03.457286 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd38c912-827f-4704-88d7-9c235f880ba7 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:03.537630 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:03.551151 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1346dfb7-481e-4bae-8657-9d30dcf1c264 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:03.553751 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:03.555980 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd38c912-827f-4704-88d7-9c235f880ba7 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:03.589074 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d327c6d-9a50-415a-9813-7c39700f7a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:03.590254 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58db95-d96a-4bb8-875c-d36e6de0c798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:03.590571 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d327c6d-9a50-415a-9813-7c39700f7a8d None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:03.590872 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58db95-d96a-4bb8-875c-d36e6de0c798 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:03.590985 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d327c6d-9a50-415a-9813-7c39700f7a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:03.591149 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d327c6d-9a50-415a-9813-7c39700f7a8d None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:03.591240 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58db95-d96a-4bb8-875c-d36e6de0c798 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:03.591414 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58db95-d96a-4bb8-875c-d36e6de0c798 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:03.591980 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d327c6d-9a50-415a-9813-7c39700f7a8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:03.592282 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d327c6d-9a50-415a-9813-7c39700f7a8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:03.593689 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d327c6d-9a50-415a-9813-7c39700f7a8d None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:03.605299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 8/15] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 13:59:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:03.610948 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d327c6d-9a50-415a-9813-7c39700f7a8d None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:03.661833 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc58db95-d96a-4bb8-875c-d36e6de0c798 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:03.663248 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc58db95-d96a-4bb8-875c-d36e6de0c798 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:03.708084 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d327c6d-9a50-415a-9813-7c39700f7a8d None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:03.740821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0957ae-379c-4ceb-9ad8-f0a6a33783fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:03.741680 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0957ae-379c-4ceb-9ad8-f0a6a33783fa None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:03.741954 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0957ae-379c-4ceb-9ad8-f0a6a33783fa None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:03.742196 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:03.742298 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0957ae-379c-4ceb-9ad8-f0a6a33783fa None admin] PATH_INFO: `/v3/users/e46b19c26ed14ce08997d6b7fd3a2efb` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:03.743976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0957ae-379c-4ceb-9ad8-f0a6a33783fa None admin] RBAC: Authorizing `identity:get_user(user_id=e46b19c26ed14ce08997d6b7fd3a2efb)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:03.746145 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0957ae-379c-4ceb-9ad8-f0a6a33783fa None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:03.747891 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 9/16] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 13:59:03 2022] GET /identity/v3/users/e46b19c26ed14ce08997d6b7fd3a2efb => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 13:59:03.753083 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0957ae-379c-4ceb-9ad8-f0a6a33783fa None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:03.762012 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 8/17] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 13:59:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 12 13:59:03.767338 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58db95-d96a-4bb8-875c-d36e6de0c798 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:03.787275 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:03.857007 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0957ae-379c-4ceb-9ad8-f0a6a33783fa None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:03.865701 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58db95-d96a-4bb8-875c-d36e6de0c798 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:03.889445 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6746705d-cde1-4b5f-a209-1217ec765115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:03.890071 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6746705d-cde1-4b5f-a209-1217ec765115 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:03.890344 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6746705d-cde1-4b5f-a209-1217ec765115 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:03.890612 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6746705d-cde1-4b5f-a209-1217ec765115 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:03.891751 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6746705d-cde1-4b5f-a209-1217ec765115 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:03.893179 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6746705d-cde1-4b5f-a209-1217ec765115 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:03.894714 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 10/18] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 13:59:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 13:59:03.898066 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b41e2-66e6-419e-9648-4a030993eab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:03.898694 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b41e2-66e6-419e-9648-4a030993eab1 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:03.898998 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b41e2-66e6-419e-9648-4a030993eab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:03.899269 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b41e2-66e6-419e-9648-4a030993eab1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:03.900374 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6746705d-cde1-4b5f-a209-1217ec765115 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:03.909856 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b41e2-66e6-419e-9648-4a030993eab1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:03.911350 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b41e2-66e6-419e-9648-4a030993eab1 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:03.928099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-0b0b41e2-66e6-419e-9648-4a030993eab1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 13:59:03.928099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:03.928099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:03.928099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:03.928099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:03.928099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:03.928099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:03.928099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:03.928099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:03.928099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:03.928099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:03.928099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:03.928099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 13:59:03.928099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 13:59:03.928099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 13:59:03.928099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 13:59:03.929668 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:03.929668 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:03.929668 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:03.929668 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:03.929668 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:03.929668 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:03.929668 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:03.929668 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:03.929668 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:03.929668 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:03.929668 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:03.929668 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:03.929668 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:03.929668 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:03.929668 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:03.929668 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:03.929668 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 13:59:03.930874 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 13:59:03.930874 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 13:59:03.930874 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 13:59:03.930874 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 9/19] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 13:59:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 13:59:03.934701 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b41e2-66e6-419e-9648-4a030993eab1 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:03.968780 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:03.998502 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6746705d-cde1-4b5f-a209-1217ec765115 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:04.000012 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:04.032510 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc052aa-60b0-49f8-9ec6-2816e8755be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:04.033116 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc052aa-60b0-49f8-9ec6-2816e8755be0 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:04.033393 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc052aa-60b0-49f8-9ec6-2816e8755be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:04.033663 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc052aa-60b0-49f8-9ec6-2816e8755be0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:04.034387 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc052aa-60b0-49f8-9ec6-2816e8755be0 None admin] RBAC: Adding query filter params (role.id=e14564f00daf48a28d4e32da8cd2efad, scope.domain.id=default, user.id=e46b19c26ed14ce08997d6b7fd3a2efb) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:04.034673 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b41e2-66e6-419e-9648-4a030993eab1 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:04.034770 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc052aa-60b0-49f8-9ec6-2816e8755be0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:04.036142 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc052aa-60b0-49f8-9ec6-2816e8755be0 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:04.045108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 11/20] 10.222.0.42 () {56 vars in 1322 bytes} [Mon Sep 12 13:59:03 2022] GET /identity/v3/role_assignments?user.id=e46b19c26ed14ce08997d6b7fd3a2efb&scope.domain.id=default&role.id=e14564f00daf48a28d4e32da8cd2efad => generated 233 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 13:59:04.074006 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b7ace-8775-49c6-9ec1-7247d0f2fddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:04.074638 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b7ace-8775-49c6-9ec1-7247d0f2fddb None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:04.074946 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b7ace-8775-49c6-9ec1-7247d0f2fddb None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:04.075220 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b7ace-8775-49c6-9ec1-7247d0f2fddb None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:04.075884 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b7ace-8775-49c6-9ec1-7247d0f2fddb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:04.076187 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b7ace-8775-49c6-9ec1-7247d0f2fddb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:04.077609 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b7ace-8775-49c6-9ec1-7247d0f2fddb None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:04.097132 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 10/21] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 13:59:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 13:59:06.260522 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ca6a6-395c-471e-891e-548065aac962 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:06.261028 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ca6a6-395c-471e-891e-548065aac962 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:06.261182 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ca6a6-395c-471e-891e-548065aac962 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:06.263366 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 12/22] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:06.278437 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40030539-84e0-40e2-8c50-f0ce29eb8202 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:06.278951 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40030539-84e0-40e2-8c50-f0ce29eb8202 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:06.279025 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40030539-84e0-40e2-8c50-f0ce29eb8202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:06.303827 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-40030539-84e0-40e2-8c50-f0ce29eb8202 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:06.305041 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-40030539-84e0-40e2-8c50-f0ce29eb8202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:06.393085 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 11/23] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:06.925846 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ca6a6-395c-471e-891e-548065aac962 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:07.014785 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41017a-480b-4651-8e3a-20a8921f8b24 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:07.015439 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41017a-480b-4651-8e3a-20a8921f8b24 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:07.015912 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41017a-480b-4651-8e3a-20a8921f8b24 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:07.017474 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 12/24] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:07.039397 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ca6a6-395c-471e-891e-548065aac962 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:07.040603 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298ffac-4451-4670-8968-09465c47d820 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:07.041088 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298ffac-4451-4670-8968-09465c47d820 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:07.041573 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298ffac-4451-4670-8968-09465c47d820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:07.086736 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c9d598-d109-4c51-b0ea-6c4c46ad6bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:07.087369 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c9d598-d109-4c51-b0ea-6c4c46ad6bec None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:07.087656 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c9d598-d109-4c51-b0ea-6c4c46ad6bec None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:07.087930 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c9d598-d109-4c51-b0ea-6c4c46ad6bec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:07.088655 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-d298ffac-4451-4670-8968-09465c47d820 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:07.090607 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-d298ffac-4451-4670-8968-09465c47d820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:07.106863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c9d598-d109-4c51-b0ea-6c4c46ad6bec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:07.116888 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:07.123596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-f2c9d598-d109-4c51-b0ea-6c4c46ad6bec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 13:59:07.123596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:07.123596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:07.123596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:07.123596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:07.123596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:07.123596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:07.123596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:07.123596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:07.123596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:07.123596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:07.123596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:07.123596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:07.123596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:07.123596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:07.123596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:07.124651 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:07.124651 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:07.124651 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:07.124651 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:07.124651 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:07.124651 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:07.124651 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:07.124651 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:07.124651 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:07.124651 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:07.124651 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:07.124651 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:07.124651 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:07.124651 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:07.124651 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:07.124651 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:07.125587 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:07.125587 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:07.125587 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:07.125587 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:07.125587 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:07.125587 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:07.125587 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:07.125587 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:07.125587 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:07.125587 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:07.125587 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 13:59:07.125587 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 13:59:07.127894 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 13/25] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 12 13:59:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 13:59:07.134863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ccc3d-e01f-4477-8264-b735bf647be7 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:07.135461 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ccc3d-e01f-4477-8264-b735bf647be7 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:07.135890 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ccc3d-e01f-4477-8264-b735bf647be7 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:07.138075 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 14/26] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:07 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:07.139005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735ccc3d-e01f-4477-8264-b735bf647be7 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:07.191253 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:07.222099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 13/27] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:07.223407 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeedaac-0204-4566-8380-0df9395b63d0 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:07.223693 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeedaac-0204-4566-8380-0df9395b63d0 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:07.223967 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeedaac-0204-4566-8380-0df9395b63d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:07.263211 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-1eeedaac-0204-4566-8380-0df9395b63d0 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:07.264417 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-1eeedaac-0204-4566-8380-0df9395b63d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:07.329004 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735ccc3d-e01f-4477-8264-b735bf647be7 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:07.387442 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059c15a-cdeb-43d7-adcf-9de79251f9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:07.387974 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059c15a-cdeb-43d7-adcf-9de79251f9a2 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:07.388252 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059c15a-cdeb-43d7-adcf-9de79251f9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:07.388837 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059c15a-cdeb-43d7-adcf-9de79251f9a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:07.390047 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d059c15a-cdeb-43d7-adcf-9de79251f9a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:07.390357 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d059c15a-cdeb-43d7-adcf-9de79251f9a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:07.391779 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d059c15a-cdeb-43d7-adcf-9de79251f9a2 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:07.406024 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 15/28] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 13:59:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:07.411857 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059c15a-cdeb-43d7-adcf-9de79251f9a2 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:07.455312 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 14/29] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:07.549744 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:07.566398 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059c15a-cdeb-43d7-adcf-9de79251f9a2 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:07.610644 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd427a58-6ea7-4d69-b480-2a85a71d5844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:07.611308 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd427a58-6ea7-4d69-b480-2a85a71d5844 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:07.611638 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd427a58-6ea7-4d69-b480-2a85a71d5844 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:07.611911 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd427a58-6ea7-4d69-b480-2a85a71d5844 None admin] PATH_INFO: `/v3/users/e46b19c26ed14ce08997d6b7fd3a2efb` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:07.613081 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd427a58-6ea7-4d69-b480-2a85a71d5844 None admin] RBAC: Authorizing `identity:get_user(user_id=e46b19c26ed14ce08997d6b7fd3a2efb)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:07.614560 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd427a58-6ea7-4d69-b480-2a85a71d5844 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:07.616157 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 16/30] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 13:59:07 2022] GET /identity/v3/users/e46b19c26ed14ce08997d6b7fd3a2efb => generated 247 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 13:59:07.621688 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeedaac-0204-4566-8380-0df9395b63d0 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:07.714885 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd427a58-6ea7-4d69-b480-2a85a71d5844 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:07.741409 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:07.766426 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeedaac-0204-4566-8380-0df9395b63d0 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:07.840098 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a4fb7e-f5b2-43e8-8c0a-9957d5c91a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:07.840860 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4fb7e-f5b2-43e8-8c0a-9957d5c91a6e None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:07.841142 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4fb7e-f5b2-43e8-8c0a-9957d5c91a6e None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:07.841410 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4fb7e-f5b2-43e8-8c0a-9957d5c91a6e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:07.843555 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a4fb7e-f5b2-43e8-8c0a-9957d5c91a6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:07.845024 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a4fb7e-f5b2-43e8-8c0a-9957d5c91a6e None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:07.846807 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 15/31] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 13:59:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:07.852533 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a4fb7e-f5b2-43e8-8c0a-9957d5c91a6e None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:07.935263 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd427a58-6ea7-4d69-b480-2a85a71d5844 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:07.972522 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf3ba9-f93a-4567-b4cd-a1ebb788e02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:07.973135 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf3ba9-f93a-4567-b4cd-a1ebb788e02b None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:07.973413 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf3ba9-f93a-4567-b4cd-a1ebb788e02b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:07.973686 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf3ba9-f93a-4567-b4cd-a1ebb788e02b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:07.984030 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a4fb7e-f5b2-43e8-8c0a-9957d5c91a6e None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:07.994460 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf3ba9-f93a-4567-b4cd-a1ebb788e02b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:07.996669 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf3ba9-f93a-4567-b4cd-a1ebb788e02b None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:08.014161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-cccf3ba9-f93a-4567-b4cd-a1ebb788e02b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 13:59:08.014161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:08.014161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:08.014161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:08.014161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:08.014161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:08.014161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:08.014161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:08.014161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:08.014161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:08.014161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:08.014161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:08.014161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 13:59:08.014161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 13:59:08.014161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 13:59:08.014161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 13:59:08.015701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:08.015701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:08.015701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:08.015701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:08.015701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:08.015701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:08.015701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:08.015701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:08.015701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:08.015701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:08.015701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:08.015701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:08.015701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:08.015701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:08.015701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:08.015701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:08.015701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 13:59:08.017707 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 13:59:08.017707 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 13:59:08.017707 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 13:59:08.017707 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 17/32] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 13:59:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 13:59:08.017707 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf3ba9-f93a-4567-b4cd-a1ebb788e02b None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:08.028082 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b2cc3-a85f-471a-ac98-d3139dc77483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:08.029085 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b2cc3-a85f-471a-ac98-d3139dc77483 None admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:08.029688 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b2cc3-a85f-471a-ac98-d3139dc77483 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:08.030024 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b2cc3-a85f-471a-ac98-d3139dc77483 None admin] PATH_INFO: `/v3/domains/default/users/e46b19c26ed14ce08997d6b7fd3a2efb/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:08.032059 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:08.032776 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7b2cc3-a85f-471a-ac98-d3139dc77483 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=e46b19c26ed14ce08997d6b7fd3a2efb, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:08.034383 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7b2cc3-a85f-471a-ac98-d3139dc77483 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:08.120878 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf3ba9-f93a-4567-b4cd-a1ebb788e02b None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:08.154037 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e6d93-68dc-45ef-95c5-4100c257d9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:08.154734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e6d93-68dc-45ef-95c5-4100c257d9dd None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:08.155109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e6d93-68dc-45ef-95c5-4100c257d9dd None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:08.155390 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e6d93-68dc-45ef-95c5-4100c257d9dd None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:08.156111 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e6d93-68dc-45ef-95c5-4100c257d9dd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:08.157491 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e6d93-68dc-45ef-95c5-4100c257d9dd None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:08.159178 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 16/33] 10.222.0.42 () {58 vars in 1361 bytes} [Mon Sep 12 13:59:07 2022] PUT /identity/v3/domains/default/users/e46b19c26ed14ce08997d6b7fd3a2efb/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 13:59:08.161129 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f4684-f7e0-45f6-a085-ff12412aeebc None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:08.161427 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f4684-f7e0-45f6-a085-ff12412aeebc None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:08.161695 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f4684-f7e0-45f6-a085-ff12412aeebc None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:08.170587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 17/34] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:08.170587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f4684-f7e0-45f6-a085-ff12412aeebc None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:08.205281 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:08.303336 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 18/35] 10.222.0.42 () {60 vars in 1131 bytes} [Mon Sep 12 13:59:08 2022] POST /identity/v3/roles => generated 219 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 13:59:08.305081 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb21729-2df6-49bd-9b36-5123cde00c08 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:08.305379 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb21729-2df6-49bd-9b36-5123cde00c08 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:08.305665 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb21729-2df6-49bd-9b36-5123cde00c08 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:08.307269 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 19/36] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:08.309931 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f65dd-452f-4c70-825b-b831b6753160 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:08.310224 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f65dd-452f-4c70-825b-b831b6753160 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:08.310507 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f65dd-452f-4c70-825b-b831b6753160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:08.318410 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f4684-f7e0-45f6-a085-ff12412aeebc None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:08.344771 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-f44f65dd-452f-4c70-825b-b831b6753160 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:08.347726 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f44f65dd-452f-4c70-825b-b831b6753160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:08.397176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6d615-0e0f-4c71-a398-cafb3783aa26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:08.397818 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6d615-0e0f-4c71-a398-cafb3783aa26 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:08.398366 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6d615-0e0f-4c71-a398-cafb3783aa26 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:08.398647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6d615-0e0f-4c71-a398-cafb3783aa26 None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:08.400237 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b6d615-0e0f-4c71-a398-cafb3783aa26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:08.400610 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b6d615-0e0f-4c71-a398-cafb3783aa26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:08.403139 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b6d615-0e0f-4c71-a398-cafb3783aa26 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:08.422949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 18/37] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 13:59:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:08.424394 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9022e5-8b15-48a0-85b6-694638ea92a3 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:08.424682 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9022e5-8b15-48a0-85b6-694638ea92a3 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:08.424957 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9022e5-8b15-48a0-85b6-694638ea92a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:08.451751 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-bd9022e5-8b15-48a0-85b6-694638ea92a3 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:08.453430 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-bd9022e5-8b15-48a0-85b6-694638ea92a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:08.454489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:08.463014 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 20/38] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:08.464078 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f65dd-452f-4c70-825b-b831b6753160 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:08.536994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 19/39] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:08.537498 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b320df-ed7c-49cc-82f0-d9da9ec40104 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:08.537588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b320df-ed7c-49cc-82f0-d9da9ec40104 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:08.537947 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b320df-ed7c-49cc-82f0-d9da9ec40104 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:08.543663 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 20/40] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:08.549737 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e6657-ee14-4377-85f0-8368f7493f0d None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:08.550026 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e6657-ee14-4377-85f0-8368f7493f0d None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:08.550304 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e6657-ee14-4377-85f0-8368f7493f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:08.576835 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:08.588002 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-c07e6657-ee14-4377-85f0-8368f7493f0d None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:08.589373 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-c07e6657-ee14-4377-85f0-8368f7493f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:08.589504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f65dd-452f-4c70-825b-b831b6753160 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:08.642225 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2afcbdc-603b-4a6a-a234-40b835709e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:08.642936 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afcbdc-603b-4a6a-a234-40b835709e7f None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:08.643215 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afcbdc-603b-4a6a-a234-40b835709e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:08.643487 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afcbdc-603b-4a6a-a234-40b835709e7f None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:08.644214 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2afcbdc-603b-4a6a-a234-40b835709e7f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:08.645793 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2afcbdc-603b-4a6a-a234-40b835709e7f None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:08.676409 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 21/41] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 13:59:08.699366 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 21/42] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Sep 12 13:59:08 2022] POST /identity/v3/projects => generated 301 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 13:59:08.901254 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e6657-ee14-4377-85f0-8368f7493f0d None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:08.916672 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2afcbdc-603b-4a6a-a234-40b835709e7f None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:08.994386 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:09.054424 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2afcbdc-603b-4a6a-a234-40b835709e7f None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:09.093040 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6017d94-3968-44a7-8ea8-8daeacad074a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:09.093577 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6017d94-3968-44a7-8ea8-8daeacad074a None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:09.093863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6017d94-3968-44a7-8ea8-8daeacad074a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:09.094140 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6017d94-3968-44a7-8ea8-8daeacad074a None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:09.094851 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6017d94-3968-44a7-8ea8-8daeacad074a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:09.097092 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6017d94-3968-44a7-8ea8-8daeacad074a None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:09.116265 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 22/43] 10.222.0.42 () {60 vars in 1131 bytes} [Mon Sep 12 13:59:08 2022] POST /identity/v3/roles => generated 225 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 13:59:09.117097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6017d94-3968-44a7-8ea8-8daeacad074a None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:09.154598 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e6657-ee14-4377-85f0-8368f7493f0d None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:09.206425 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:09.213456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fdeb1e-f70a-4272-aee3-d47645b9743c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:09.215612 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fdeb1e-f70a-4272-aee3-d47645b9743c None admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:09.216271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fdeb1e-f70a-4272-aee3-d47645b9743c None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:09.216800 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fdeb1e-f70a-4272-aee3-d47645b9743c None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:09.217721 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fdeb1e-f70a-4272-aee3-d47645b9743c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:09.219626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fdeb1e-f70a-4272-aee3-d47645b9743c None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:09.239220 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6017d94-3968-44a7-8ea8-8daeacad074a None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:09.239632 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 22/44] 10.222.0.42 () {60 vars in 1131 bytes} [Mon Sep 12 13:59:08 2022] POST /identity/v3/roles => generated 223 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 13:59:09.255397 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:09.282269 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d718e85e-ef41-4b6c-b857-72913390d544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:09.282974 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718e85e-ef41-4b6c-b857-72913390d544 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:09.283259 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718e85e-ef41-4b6c-b857-72913390d544 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:09.283537 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718e85e-ef41-4b6c-b857-72913390d544 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:09.284739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d718e85e-ef41-4b6c-b857-72913390d544 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:09.286198 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d718e85e-ef41-4b6c-b857-72913390d544 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:09.288186 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 23/45] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 13:59:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:09.293800 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fdeb1e-f70a-4272-aee3-d47645b9743c None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:09.417770 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fdeb1e-f70a-4272-aee3-d47645b9743c None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:09.456372 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f4039-b8f0-46ee-a623-046f33596a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:09.456988 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f4039-b8f0-46ee-a623-046f33596a7c None admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:09.457274 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f4039-b8f0-46ee-a623-046f33596a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:09.457554 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f4039-b8f0-46ee-a623-046f33596a7c None admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:09.458285 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641f4039-b8f0-46ee-a623-046f33596a7c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:09.459803 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641f4039-b8f0-46ee-a623-046f33596a7c None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:09.489388 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 23/46] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Sep 12 13:59:09 2022] POST /identity/v3/projects => generated 312 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 13:59:10.600619 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a86de-84f5-40df-8ff2-7971c6c7e51b None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:10.601149 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a86de-84f5-40df-8ff2-7971c6c7e51b None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:10.601256 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a86de-84f5-40df-8ff2-7971c6c7e51b None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:10.602514 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 24/47] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:10.607908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d8073-7e5a-4efe-ad7e-3019dff943b8 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:10.608322 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d8073-7e5a-4efe-ad7e-3019dff943b8 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:10.608457 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d8073-7e5a-4efe-ad7e-3019dff943b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:10.626657 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-5a2d8073-7e5a-4efe-ad7e-3019dff943b8 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:10.627870 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-5a2d8073-7e5a-4efe-ad7e-3019dff943b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:10.666431 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:10.693808 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 24/48] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:10.958553 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a86de-84f5-40df-8ff2-7971c6c7e51b None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:11.056423 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a86de-84f5-40df-8ff2-7971c6c7e51b None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:11.088821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729afc3f-d92b-461d-ad86-6aab882f6246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:11.089455 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729afc3f-d92b-461d-ad86-6aab882f6246 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:11.089740 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729afc3f-d92b-461d-ad86-6aab882f6246 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:11.090015 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729afc3f-d92b-461d-ad86-6aab882f6246 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:11.099794 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729afc3f-d92b-461d-ad86-6aab882f6246 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:11.105617 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-729afc3f-d92b-461d-ad86-6aab882f6246 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 13:59:11.105617 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:11.105617 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:11.105617 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:11.105617 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:11.105617 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:11.105617 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:11.105617 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:11.105617 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:11.105617 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:11.105617 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:11.105617 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:11.105617 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:11.105617 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:11.105617 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:11.105617 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:11.106439 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:11.106439 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:11.106439 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:11.106439 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:11.106439 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:11.106439 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:11.106439 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:11.106439 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:11.106439 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:11.106439 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:11.106439 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:11.106439 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:11.106439 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:11.106439 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:11.106439 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:11.106439 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:11.107614 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:11.107614 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:11.107614 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:11.107614 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:11.107614 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:11.107614 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:11.107614 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:11.107614 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:11.107614 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:11.107614 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:11.107614 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 13:59:11.107614 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 13:59:11.107614 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 25/49] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 12 13:59:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 13:59:11.112687 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d8073-7e5a-4efe-ad7e-3019dff943b8 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:11.209403 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d8073-7e5a-4efe-ad7e-3019dff943b8 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:11.242712 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6548d785-d8c5-462c-8ddb-de4b7fdd80e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:11.243347 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6548d785-d8c5-462c-8ddb-de4b7fdd80e7 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:11.243625 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6548d785-d8c5-462c-8ddb-de4b7fdd80e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:11.243895 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6548d785-d8c5-462c-8ddb-de4b7fdd80e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:11.244560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6548d785-d8c5-462c-8ddb-de4b7fdd80e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:11.244859 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6548d785-d8c5-462c-8ddb-de4b7fdd80e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:11.246875 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6548d785-d8c5-462c-8ddb-de4b7fdd80e7 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:11.258332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 25/50] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 13:59:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:11.264023 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729afc3f-d92b-461d-ad86-6aab882f6246 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:11.303314 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:11.362105 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729afc3f-d92b-461d-ad86-6aab882f6246 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:11.394763 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63881f27-476e-45a2-8167-695002b217c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:11.395421 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63881f27-476e-45a2-8167-695002b217c8 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:11.395713 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63881f27-476e-45a2-8167-695002b217c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:11.395969 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63881f27-476e-45a2-8167-695002b217c8 None admin] PATH_INFO: `/v3/users/e46b19c26ed14ce08997d6b7fd3a2efb` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:11.397071 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63881f27-476e-45a2-8167-695002b217c8 None admin] RBAC: Authorizing `identity:get_user(user_id=e46b19c26ed14ce08997d6b7fd3a2efb)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:11.398500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63881f27-476e-45a2-8167-695002b217c8 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:11.400092 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 26/51] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 13:59:11 2022] GET /identity/v3/users/e46b19c26ed14ce08997d6b7fd3a2efb => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:11.406073 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6548d785-d8c5-462c-8ddb-de4b7fdd80e7 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:11.443138 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:11.508143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6548d785-d8c5-462c-8ddb-de4b7fdd80e7 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:11.544793 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb0349-9a62-488d-8360-05aff2054d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:11.545437 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb0349-9a62-488d-8360-05aff2054d70 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:11.545716 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb0349-9a62-488d-8360-05aff2054d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:11.545994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb0349-9a62-488d-8360-05aff2054d70 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:11.547149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfb0349-9a62-488d-8360-05aff2054d70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:11.548619 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfb0349-9a62-488d-8360-05aff2054d70 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:11.550433 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 26/52] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 13:59:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:11.555965 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63881f27-476e-45a2-8167-695002b217c8 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:11.638313 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54ec18-d079-4c6d-95ea-26459e246d5a None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:11.638871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54ec18-d079-4c6d-95ea-26459e246d5a None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:11.638989 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54ec18-d079-4c6d-95ea-26459e246d5a None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:11.640316 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 27/53] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 13:59:11.645149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed7250b-e523-41a6-8cce-ea9ff587173a None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:11.645425 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed7250b-e523-41a6-8cce-ea9ff587173a None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:11.645693 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed7250b-e523-41a6-8cce-ea9ff587173a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:11.655458 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63881f27-476e-45a2-8167-695002b217c8 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:11.662717 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:11.663942 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-6ed7250b-e523-41a6-8cce-ea9ff587173a None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:11.665137 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-6ed7250b-e523-41a6-8cce-ea9ff587173a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:11.690388 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed71797d-db6b-498e-99f5-d098dcb8ab6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:11.691031 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed71797d-db6b-498e-99f5-d098dcb8ab6f None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:11.691319 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed71797d-db6b-498e-99f5-d098dcb8ab6f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:11.691588 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed71797d-db6b-498e-99f5-d098dcb8ab6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:11.692304 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed71797d-db6b-498e-99f5-d098dcb8ab6f None admin] RBAC: Adding query filter params (role.id=e14564f00daf48a28d4e32da8cd2efad, scope.domain.id=default, user.id=e46b19c26ed14ce08997d6b7fd3a2efb) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:11.692608 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed71797d-db6b-498e-99f5-d098dcb8ab6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:11.694014 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed71797d-db6b-498e-99f5-d098dcb8ab6f None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:11.700542 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 27/54] 10.222.0.42 () {56 vars in 1322 bytes} [Mon Sep 12 13:59:11 2022] GET /identity/v3/role_assignments?user.id=e46b19c26ed14ce08997d6b7fd3a2efb&scope.domain.id=default&role.id=e14564f00daf48a28d4e32da8cd2efad => generated 530 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:11.736229 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 28/55] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 13:59:12.007174 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed71797d-db6b-498e-99f5-d098dcb8ab6f None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:12.115097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed71797d-db6b-498e-99f5-d098dcb8ab6f None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:12.147206 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2054474-2aaa-4ccc-b350-e1889bd18412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:12.147822 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2054474-2aaa-4ccc-b350-e1889bd18412 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:12.148104 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2054474-2aaa-4ccc-b350-e1889bd18412 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:12.148375 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2054474-2aaa-4ccc-b350-e1889bd18412 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:12.149458 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2054474-2aaa-4ccc-b350-e1889bd18412 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:12.150933 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2054474-2aaa-4ccc-b350-e1889bd18412 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:12.152471 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 28/56] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 13:59:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:12.158004 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed7250b-e523-41a6-8cce-ea9ff587173a None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:12.190053 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:12.254129 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed7250b-e523-41a6-8cce-ea9ff587173a None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:12.286775 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf156a9-2103-4ac1-af58-6b75298d2f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:12.287405 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf156a9-2103-4ac1-af58-6b75298d2f7c None admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:12.287686 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf156a9-2103-4ac1-af58-6b75298d2f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:12.287959 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf156a9-2103-4ac1-af58-6b75298d2f7c None admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:12.288605 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf156a9-2103-4ac1-af58-6b75298d2f7c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:12.290051 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf156a9-2103-4ac1-af58-6b75298d2f7c None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:12.319187 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 29/57] 10.222.0.42 () {60 vars in 1140 bytes} [Mon Sep 12 13:59:12 2022] POST /identity/v3/projects => generated 298 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 13:59:14.478091 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c2a4d-d152-4e3b-baed-6110333fb194 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:14.478701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c2a4d-d152-4e3b-baed-6110333fb194 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:14.478701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c2a4d-d152-4e3b-baed-6110333fb194 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:14.480055 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 29/58] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:14.485850 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb3d2c-02d0-4e0d-a8bd-58d659592419 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:14.486281 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb3d2c-02d0-4e0d-a8bd-58d659592419 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:14.486406 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb3d2c-02d0-4e0d-a8bd-58d659592419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:14.504143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-c9fb3d2c-02d0-4e0d-a8bd-58d659592419 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:14.505328 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-c9fb3d2c-02d0-4e0d-a8bd-58d659592419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:14.569182 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 30/59] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:14.841113 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c2a4d-d152-4e3b-baed-6110333fb194 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:14.981710 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:15.039865 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c2a4d-d152-4e3b-baed-6110333fb194 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:15.072107 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b43cce7-2b06-4851-9101-c96efc6e9e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:15.072725 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43cce7-2b06-4851-9101-c96efc6e9e87 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:15.073012 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43cce7-2b06-4851-9101-c96efc6e9e87 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:15.073857 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43cce7-2b06-4851-9101-c96efc6e9e87 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:15.075000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b43cce7-2b06-4851-9101-c96efc6e9e87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:15.076414 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b43cce7-2b06-4851-9101-c96efc6e9e87 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:15.077953 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 30/60] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 13:59:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:15.083553 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb3d2c-02d0-4e0d-a8bd-58d659592419 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:15.100147 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:15.179057 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb3d2c-02d0-4e0d-a8bd-58d659592419 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:15.211904 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd8097-8a81-418b-83cf-1d7914082b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:15.212497 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd8097-8a81-418b-83cf-1d7914082b29 None admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:15.212773 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd8097-8a81-418b-83cf-1d7914082b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:15.213038 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd8097-8a81-418b-83cf-1d7914082b29 None admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:15.213682 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fd8097-8a81-418b-83cf-1d7914082b29 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:15.215172 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fd8097-8a81-418b-83cf-1d7914082b29 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:15.240352 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 31/61] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Sep 12 13:59:15 2022] POST /identity/v3/users => generated 275 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 13:59:18.362669 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b9411-3dfb-4949-9125-06033909c78a None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:18.363744 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b9411-3dfb-4949-9125-06033909c78a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:18.364251 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b9411-3dfb-4949-9125-06033909c78a None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:18.365862 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 31/62] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:18.371958 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5768da0-1868-4a7e-88b7-e185b8ebe2d6 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:18.372762 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5768da0-1868-4a7e-88b7-e185b8ebe2d6 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:18.373233 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5768da0-1868-4a7e-88b7-e185b8ebe2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:18.395901 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-b5768da0-1868-4a7e-88b7-e185b8ebe2d6 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:18.397421 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-b5768da0-1868-4a7e-88b7-e185b8ebe2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:18.487035 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 32/63] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:18.810654 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb93865-ae2f-4cc5-a747-291b4e16d93a None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:18.811378 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb93865-ae2f-4cc5-a747-291b4e16d93a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:18.811867 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb93865-ae2f-4cc5-a747-291b4e16d93a None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:18.813654 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 32/64] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:18.819343 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b76d03-c403-4f49-8928-553c0bdc28ac None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:18.819878 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b76d03-c403-4f49-8928-553c0bdc28ac None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:18.820345 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b76d03-c403-4f49-8928-553c0bdc28ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:18.833991 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:18.850003 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-94b76d03-c403-4f49-8928-553c0bdc28ac None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:18.855392 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-94b76d03-c403-4f49-8928-553c0bdc28ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:18.869677 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578da06-c76c-4f6f-913a-a9544ebc5d4f None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:18.870824 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578da06-c76c-4f6f-913a-a9544ebc5d4f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:18.871346 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578da06-c76c-4f6f-913a-a9544ebc5d4f None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:18.872962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 33/65] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:18.878660 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d64deb-18b6-4537-8712-fbc021560235 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:18.879389 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d64deb-18b6-4537-8712-fbc021560235 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:18.879860 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d64deb-18b6-4537-8712-fbc021560235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:18.898955 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-44d64deb-18b6-4537-8712-fbc021560235 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:18.900522 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-44d64deb-18b6-4537-8712-fbc021560235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:18.950366 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 33/66] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:18.958860 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b76d03-c403-4f49-8928-553c0bdc28ac None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:18.999531 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 34/67] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 13:59:19.004987 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089d72b-1e9c-4fc0-978a-143ed713f5ea None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:19.005269 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089d72b-1e9c-4fc0-978a-143ed713f5ea None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:19.005552 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089d72b-1e9c-4fc0-978a-143ed713f5ea None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:19.006866 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 35/68] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:19.013451 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b33639-f587-4b84-ab5f-4812bf17e3d0 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:19.013740 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b33639-f587-4b84-ab5f-4812bf17e3d0 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:19.014020 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b33639-f587-4b84-ab5f-4812bf17e3d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:19.051821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-b3b33639-f587-4b84-ab5f-4812bf17e3d0 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:19.056187 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b3b33639-f587-4b84-ab5f-4812bf17e3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:19.112534 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:19.138575 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b76d03-c403-4f49-8928-553c0bdc28ac None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:19.187723 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 36/69] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:19.207216 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:19.213333 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a4b71-6ee8-4d30-8a02-01acac5a6b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:19.214232 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a4b71-6ee8-4d30-8a02-01acac5a6b2d None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:19.218032 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a4b71-6ee8-4d30-8a02-01acac5a6b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:19.218309 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a4b71-6ee8-4d30-8a02-01acac5a6b2d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:19.222577 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b33639-f587-4b84-ab5f-4812bf17e3d0 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:19.234870 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a4b71-6ee8-4d30-8a02-01acac5a6b2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:19.236029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-996a4b71-6ee8-4d30-8a02-01acac5a6b2d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 13:59:19.236029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:19.236029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:19.236029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:19.236029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:19.236029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:19.236029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:19.236029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:19.236029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:19.236029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:19.236029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:19.236029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:19.236029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:19.236029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:19.236029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:19.236029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:19.236838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:19.236838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:19.236838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:19.236838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:19.236838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:19.236838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:19.236838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:19.236838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:19.236838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:19.236838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:19.236838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:19.236838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:19.236838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:19.236838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:19.236838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:19.236838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:19.237593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:19.237593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:19.237593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:19.237593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:19.237593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:19.237593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:19.237593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:19.237593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:19.237593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:19.237593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:19.237593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 13:59:19.237593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 13:59:19.237593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 34/70] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 13:59:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 13:59:19.248338 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a4b71-6ee8-4d30-8a02-01acac5a6b2d None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:19.356915 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a4b71-6ee8-4d30-8a02-01acac5a6b2d None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:19.369196 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b33639-f587-4b84-ab5f-4812bf17e3d0 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:19.394191 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9f08b-0ef9-4f1b-9de1-08ab9bd2e047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:19.394794 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9f08b-0ef9-4f1b-9de1-08ab9bd2e047 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:19.395095 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9f08b-0ef9-4f1b-9de1-08ab9bd2e047 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:19.395366 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9f08b-0ef9-4f1b-9de1-08ab9bd2e047 None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:19.396027 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b9f08b-0ef9-4f1b-9de1-08ab9bd2e047 None admin] RBAC: Adding query filter params (name=member) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:19.396326 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b9f08b-0ef9-4f1b-9de1-08ab9bd2e047 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:19.397781 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b9f08b-0ef9-4f1b-9de1-08ab9bd2e047 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:19.406194 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 35/71] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 12 13:59:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 13:59:19.407890 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a696bce4-b209-493d-9220-a2b9a11cca3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:19.408774 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a696bce4-b209-493d-9220-a2b9a11cca3a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:19.409248 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a696bce4-b209-493d-9220-a2b9a11cca3a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:19.409712 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a696bce4-b209-493d-9220-a2b9a11cca3a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:19.411910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63ce8a-3af5-4c0b-9d38-522f5b3eb7ee None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:19.412197 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63ce8a-3af5-4c0b-9d38-522f5b3eb7ee None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:19.412475 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63ce8a-3af5-4c0b-9d38-522f5b3eb7ee None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:19.413708 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 36/72] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:19.414401 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b63ce8a-3af5-4c0b-9d38-522f5b3eb7ee None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:19.423146 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a696bce4-b209-493d-9220-a2b9a11cca3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:19.427997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-a696bce4-b209-493d-9220-a2b9a11cca3a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 13:59:19.427997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:19.427997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:19.427997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:19.427997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:19.427997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:19.427997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:19.427997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:19.427997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:19.427997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:19.427997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:19.427997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:19.427997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:19.427997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:19.427997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:19.427997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:19.428845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:19.428845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:19.428845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:19.428845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:19.428845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:19.428845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:19.428845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:19.428845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:19.428845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:19.428845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:19.428845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:19.428845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:19.428845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:19.428845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:19.428845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:19.428845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:19.434690 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:19.434690 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:19.434690 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:19.434690 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:19.434690 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:19.434690 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:19.434690 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:19.434690 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:19.434690 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:19.434690 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:19.434690 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 13:59:19.434690 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 13:59:19.434690 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 37/73] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 12 13:59:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 13:59:19.434690 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a696bce4-b209-493d-9220-a2b9a11cca3a None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:19.460992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:19.479336 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:19.523781 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b63ce8a-3af5-4c0b-9d38-522f5b3eb7ee None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:19.534857 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a696bce4-b209-493d-9220-a2b9a11cca3a None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:19.555804 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da7ed45-8608-4bf1-92e3-a27b294cabd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:19.556422 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7ed45-8608-4bf1-92e3-a27b294cabd5 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:19.556699 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7ed45-8608-4bf1-92e3-a27b294cabd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:19.556971 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7ed45-8608-4bf1-92e3-a27b294cabd5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:19.567203 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da7ed45-8608-4bf1-92e3-a27b294cabd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:19.567648 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d13d40-ef32-407b-872a-04753f5bcd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:19.568275 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d13d40-ef32-407b-872a-04753f5bcd84 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:19.568559 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d13d40-ef32-407b-872a-04753f5bcd84 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:19.568836 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d13d40-ef32-407b-872a-04753f5bcd84 None admin] PATH_INFO: `/v3/users/96b89bc850f04d96bedaea747c2c46b7` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:19.572691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-8da7ed45-8608-4bf1-92e3-a27b294cabd5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 13:59:19.572691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:19.572691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:19.572691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:19.572691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:19.572691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:19.572691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:19.572691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:19.572691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:19.572691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:19.572691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:19.572691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:19.572691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:19.572691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:19.572691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:19.572691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:19.574087 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:19.574087 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:19.574087 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:19.574087 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:19.574087 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:19.574087 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:19.574087 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:19.574087 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:19.574087 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:19.574087 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:19.574087 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:19.574087 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:19.574087 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:19.574087 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:19.574087 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:19.574087 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:19.574831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:19.574831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:19.574831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:19.574831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:19.574831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:19.574831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:19.574831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:19.574831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:19.574831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:19.574831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:19.574831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 13:59:19.574831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 13:59:19.574831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 37/74] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 12 13:59:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 13:59:19.576016 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe8dcf-68e9-433e-b06e-6100a794dc66 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:19.576016 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe8dcf-68e9-433e-b06e-6100a794dc66 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:19.576016 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe8dcf-68e9-433e-b06e-6100a794dc66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:19.587081 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d13d40-ef32-407b-872a-04753f5bcd84 None admin] RBAC: Authorizing `identity:get_user(user_id=96b89bc850f04d96bedaea747c2c46b7)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:19.588510 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d13d40-ef32-407b-872a-04753f5bcd84 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:19.590400 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 38/75] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 13:59:19 2022] GET /identity/v3/users/96b89bc850f04d96bedaea747c2c46b7 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:19.591207 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d13d40-ef32-407b-872a-04753f5bcd84 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:19.594249 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-a1fe8dcf-68e9-433e-b06e-6100a794dc66 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:19.595570 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-a1fe8dcf-68e9-433e-b06e-6100a794dc66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:19.668785 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 38/76] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:19.669536 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe8dcf-68e9-433e-b06e-6100a794dc66 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:19.689560 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:19.707809 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d13d40-ef32-407b-872a-04753f5bcd84 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:19.717451 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:19.742689 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718feba5-d839-4e33-b0b2-54908b4f7bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:19.743325 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718feba5-d839-4e33-b0b2-54908b4f7bea None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:19.743601 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718feba5-d839-4e33-b0b2-54908b4f7bea None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:19.743876 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718feba5-d839-4e33-b0b2-54908b4f7bea None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:19.744535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718feba5-d839-4e33-b0b2-54908b4f7bea None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:19.744833 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718feba5-d839-4e33-b0b2-54908b4f7bea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:19.746711 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718feba5-d839-4e33-b0b2-54908b4f7bea None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:19.755723 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 39/77] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 13:59:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:19.756663 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718feba5-d839-4e33-b0b2-54908b4f7bea None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:19.779068 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe8dcf-68e9-433e-b06e-6100a794dc66 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:19.819245 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71191e38-e270-4f41-893e-564f7d8a352f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:19.820148 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71191e38-e270-4f41-893e-564f7d8a352f None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:19.820627 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71191e38-e270-4f41-893e-564f7d8a352f None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:19.821038 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71191e38-e270-4f41-893e-564f7d8a352f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:19.832220 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71191e38-e270-4f41-893e-564f7d8a352f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:19.837124 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-71191e38-e270-4f41-893e-564f7d8a352f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 13:59:19.837124 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:19.837124 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:19.837124 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:19.837124 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:19.837124 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:19.837124 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:19.837124 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:19.837124 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:19.837124 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:19.837124 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:19.837124 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:19.837124 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:19.837124 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:19.837124 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:19.837124 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:19.838051 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:19.838051 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:19.838051 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:19.838051 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:19.838051 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:19.838051 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:19.838051 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:19.838051 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:19.838051 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:19.838051 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:19.838051 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:19.838051 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:19.838051 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:19.838051 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:19.838051 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:19.838051 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:19.838945 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:19.838945 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:19.838945 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:19.838945 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:19.838945 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:19.838945 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:19.838945 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:19.838945 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:19.838945 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:19.838945 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:19.838945 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 13:59:19.838945 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 13:59:19.838945 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 39/78] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 13:59:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 13:59:19.844309 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71191e38-e270-4f41-893e-564f7d8a352f None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:19.866849 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718feba5-d839-4e33-b0b2-54908b4f7bea None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:19.904925 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6b0113-9d86-423d-a07f-b4d500f502bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:19.905540 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b0113-9d86-423d-a07f-b4d500f502bd None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:19.905855 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b0113-9d86-423d-a07f-b4d500f502bd None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:19.906089 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b0113-9d86-423d-a07f-b4d500f502bd None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:19.906756 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6b0113-9d86-423d-a07f-b4d500f502bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:19.907091 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6b0113-9d86-423d-a07f-b4d500f502bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:19.908451 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6b0113-9d86-423d-a07f-b4d500f502bd None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:19.922127 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 40/79] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 13:59:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:19.922907 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6b0113-9d86-423d-a07f-b4d500f502bd None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:19.934504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:19.959527 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71191e38-e270-4f41-893e-564f7d8a352f None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:19.963074 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:19.996635 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab173d-9ec1-45aa-93c6-b4bb10529bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:19.997235 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab173d-9ec1-45aa-93c6-b4bb10529bea None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:19.997510 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab173d-9ec1-45aa-93c6-b4bb10529bea None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:19.997780 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab173d-9ec1-45aa-93c6-b4bb10529bea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:20.027392 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6b0113-9d86-423d-a07f-b4d500f502bd None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:20.061454 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f21078f-0481-4dca-a718-7fdfc43a0fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:20.062091 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21078f-0481-4dca-a718-7fdfc43a0fe7 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:20.062370 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21078f-0481-4dca-a718-7fdfc43a0fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:20.062709 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21078f-0481-4dca-a718-7fdfc43a0fe7 None admin] PATH_INFO: `/v3/users/96b89bc850f04d96bedaea747c2c46b7` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:20.064190 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f21078f-0481-4dca-a718-7fdfc43a0fe7 None admin] RBAC: Authorizing `identity:get_user(user_id=96b89bc850f04d96bedaea747c2c46b7)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:20.065521 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f21078f-0481-4dca-a718-7fdfc43a0fe7 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:20.067370 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 41/80] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 13:59:19 2022] GET /identity/v3/users/96b89bc850f04d96bedaea747c2c46b7 => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:20.068341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab173d-9ec1-45aa-93c6-b4bb10529bea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:20.069887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab173d-9ec1-45aa-93c6-b4bb10529bea None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:20.071676 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f21078f-0481-4dca-a718-7fdfc43a0fe7 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:20.146439 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:20.165631 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 40/81] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 13:59:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 13:59:20.166555 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab173d-9ec1-45aa-93c6-b4bb10529bea None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:20.169381 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f21078f-0481-4dca-a718-7fdfc43a0fe7 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:20.172098 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:20.203000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d0212b-d2f7-415b-a23a-fbf37a6ca86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:20.203541 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d0212b-d2f7-415b-a23a-fbf37a6ca86e None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:20.203824 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d0212b-d2f7-415b-a23a-fbf37a6ca86e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:20.204095 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d0212b-d2f7-415b-a23a-fbf37a6ca86e None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:20.204759 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d0212b-d2f7-415b-a23a-fbf37a6ca86e None admin] RBAC: Adding query filter params (name=member) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:20.205060 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d0212b-d2f7-415b-a23a-fbf37a6ca86e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:20.206406 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d0212b-d2f7-415b-a23a-fbf37a6ca86e None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:20.214640 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 42/82] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 12 13:59:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:20.215455 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d0212b-d2f7-415b-a23a-fbf37a6ca86e None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:20.272961 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab173d-9ec1-45aa-93c6-b4bb10529bea None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:20.308580 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a2af4-18ac-42f7-9fd4-a443737d699d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:20.309204 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a2af4-18ac-42f7-9fd4-a443737d699d None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:20.309479 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a2af4-18ac-42f7-9fd4-a443737d699d None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:20.309752 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a2af4-18ac-42f7-9fd4-a443737d699d None admin] PATH_INFO: `/v3/users/e46b19c26ed14ce08997d6b7fd3a2efb` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:20.310875 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a2af4-18ac-42f7-9fd4-a443737d699d None admin] RBAC: Authorizing `identity:get_user(user_id=e46b19c26ed14ce08997d6b7fd3a2efb)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:20.312324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a2af4-18ac-42f7-9fd4-a443737d699d None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:20.313891 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 41/83] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 13:59:20 2022] GET /identity/v3/users/e46b19c26ed14ce08997d6b7fd3a2efb => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:20.314655 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a2af4-18ac-42f7-9fd4-a443737d699d None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:20.315523 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d0212b-d2f7-415b-a23a-fbf37a6ca86e None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:20.349712 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a7cdb-3405-4267-ba21-ef716068853e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:20.350351 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a7cdb-3405-4267-ba21-ef716068853e None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:20.350635 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a7cdb-3405-4267-ba21-ef716068853e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:20.350940 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a7cdb-3405-4267-ba21-ef716068853e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:20.352060 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2a7cdb-3405-4267-ba21-ef716068853e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:20.353514 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2a7cdb-3405-4267-ba21-ef716068853e None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:20.355695 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 43/84] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 13:59:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:20.356461 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a7cdb-3405-4267-ba21-ef716068853e None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:20.366252 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:20.384160 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:20.413437 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a2af4-18ac-42f7-9fd4-a443737d699d None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:20.448397 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8655a2c2-7fe6-40ba-9bfc-86f2fba7bb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:20.448995 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655a2c2-7fe6-40ba-9bfc-86f2fba7bb37 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:20.449269 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655a2c2-7fe6-40ba-9bfc-86f2fba7bb37 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:20.449534 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655a2c2-7fe6-40ba-9bfc-86f2fba7bb37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:20.456060 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a7cdb-3405-4267-ba21-ef716068853e None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:20.489104 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d31f06e-a42e-42f5-991f-927b90a3bc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:20.489719 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31f06e-a42e-42f5-991f-927b90a3bc67 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:20.489997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31f06e-a42e-42f5-991f-927b90a3bc67 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:20.490268 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31f06e-a42e-42f5-991f-927b90a3bc67 None admin] PATH_INFO: `/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:20.491408 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d31f06e-a42e-42f5-991f-927b90a3bc67 None admin] RBAC: Authorizing `identity:get_project(project_id=e10c65c7a17c44a7a8711e7701efdcd0)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:20.492833 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d31f06e-a42e-42f5-991f-927b90a3bc67 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:20.494427 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 44/85] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 13:59:20 2022] GET /identity/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0 => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:20.495354 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d31f06e-a42e-42f5-991f-927b90a3bc67 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:20.519710 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8655a2c2-7fe6-40ba-9bfc-86f2fba7bb37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:20.521162 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8655a2c2-7fe6-40ba-9bfc-86f2fba7bb37 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:20.567110 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:20.587113 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:20.594817 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d31f06e-a42e-42f5-991f-927b90a3bc67 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:20.620874 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 42/86] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 13:59:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 13:59:20.621730 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8655a2c2-7fe6-40ba-9bfc-86f2fba7bb37 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:20.631746 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f44d4-d944-4577-a220-efb7ed958294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:20.632382 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f44d4-d944-4577-a220-efb7ed958294 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:20.632662 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f44d4-d944-4577-a220-efb7ed958294 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:20.632941 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f44d4-d944-4577-a220-efb7ed958294 None admin] PATH_INFO: `/v3/users/96b89bc850f04d96bedaea747c2c46b7` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:20.634073 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4f44d4-d944-4577-a220-efb7ed958294 None admin] RBAC: Authorizing `identity:get_user(user_id=96b89bc850f04d96bedaea747c2c46b7)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:20.635531 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4f44d4-d944-4577-a220-efb7ed958294 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:20.637103 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 45/87] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 13:59:20 2022] GET /identity/v3/users/96b89bc850f04d96bedaea747c2c46b7 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:20.637980 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f44d4-d944-4577-a220-efb7ed958294 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:20.724976 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8655a2c2-7fe6-40ba-9bfc-86f2fba7bb37 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:20.734084 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f44d4-d944-4577-a220-efb7ed958294 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:20.759388 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1661896e-f3ff-470d-a0a9-643743ef5ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:20.759993 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1661896e-f3ff-470d-a0a9-643743ef5ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:20.760276 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1661896e-f3ff-470d-a0a9-643743ef5ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:20.760551 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1661896e-f3ff-470d-a0a9-643743ef5ee0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:20.766974 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab2b48-fdf5-4389-9164-ac6183527d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:20.767573 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab2b48-fdf5-4389-9164-ac6183527d6d None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:20.767856 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab2b48-fdf5-4389-9164-ac6183527d6d None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:20.768129 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab2b48-fdf5-4389-9164-ac6183527d6d None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:20.768778 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ab2b48-fdf5-4389-9164-ac6183527d6d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:20.770193 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ab2b48-fdf5-4389-9164-ac6183527d6d None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:20.791853 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:20.797517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 46/88] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Sep 12 13:59:20 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 13:59:20.798889 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab2b48-fdf5-4389-9164-ac6183527d6d None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:20.803279 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:20.844992 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1661896e-f3ff-470d-a0a9-643743ef5ee0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:20.847359 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1661896e-f3ff-470d-a0a9-643743ef5ee0 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:20.931821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab2b48-fdf5-4389-9164-ac6183527d6d None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:20.973218 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 43/89] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 13:59:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 13:59:20.973218 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1661896e-f3ff-470d-a0a9-643743ef5ee0 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:20.976980 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a877e-227b-4d45-a103-922af49388e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:20.977125 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a877e-227b-4d45-a103-922af49388e7 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:20.977343 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a877e-227b-4d45-a103-922af49388e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:20.977616 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a877e-227b-4d45-a103-922af49388e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:20.978357 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a877e-227b-4d45-a103-922af49388e7 None admin] RBAC: Adding query filter params (role.id=d0a6f7c5c34a4925b9efb7dd4ac915fc, scope.project.id=e10c65c7a17c44a7a8711e7701efdcd0, user.id=96b89bc850f04d96bedaea747c2c46b7) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:20.978663 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a877e-227b-4d45-a103-922af49388e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:20.980174 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a877e-227b-4d45-a103-922af49388e7 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:20.986497 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 47/90] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 13:59:20 2022] GET /identity/v3/role_assignments?user.id=96b89bc850f04d96bedaea747c2c46b7&scope.project.id=e10c65c7a17c44a7a8711e7701efdcd0&role.id=d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:20.987370 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a877e-227b-4d45-a103-922af49388e7 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:21.079515 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:21.081605 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:21.131515 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1661896e-f3ff-470d-a0a9-643743ef5ee0 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:21.138760 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a877e-227b-4d45-a103-922af49388e7 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:21.180093 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb8b04-59e9-4699-a171-515a39ba5173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:21.181072 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb8b04-59e9-4699-a171-515a39ba5173 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:21.181519 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb8b04-59e9-4699-a171-515a39ba5173 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:21.181790 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544a160b-0724-45c0-bcab-212e5eeafe0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:21.182098 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb8b04-59e9-4699-a171-515a39ba5173 None admin] PATH_INFO: `/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:21.183157 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a160b-0724-45c0-bcab-212e5eeafe0a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:21.183539 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a160b-0724-45c0-bcab-212e5eeafe0a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:21.183913 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a160b-0724-45c0-bcab-212e5eeafe0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:21.184412 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb8b04-59e9-4699-a171-515a39ba5173 None admin] RBAC: Authorizing `identity:get_project(project_id=e10c65c7a17c44a7a8711e7701efdcd0)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:21.186123 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb8b04-59e9-4699-a171-515a39ba5173 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:21.188216 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 44/91] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 13:59:20 2022] GET /identity/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:21.188957 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb8b04-59e9-4699-a171-515a39ba5173 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:21.263260 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544a160b-0724-45c0-bcab-212e5eeafe0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:21.264965 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544a160b-0724-45c0-bcab-212e5eeafe0a None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:21.304213 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb8b04-59e9-4699-a171-515a39ba5173 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:21.336637 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:21.357396 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0351fa5-d53a-4bdc-9239-b0f6affe2a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:21.358038 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0351fa5-d53a-4bdc-9239-b0f6affe2a32 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:21.358314 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0351fa5-d53a-4bdc-9239-b0f6affe2a32 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:21.358585 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0351fa5-d53a-4bdc-9239-b0f6affe2a32 None admin] PATH_INFO: `/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:21.364296 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0351fa5-d53a-4bdc-9239-b0f6affe2a32 None admin] RBAC: Authorizing `identity:get_project(project_id=e10c65c7a17c44a7a8711e7701efdcd0)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:21.365757 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0351fa5-d53a-4bdc-9239-b0f6affe2a32 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:21.367678 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 45/92] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 13:59:21 2022] GET /identity/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:21.368462 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0351fa5-d53a-4bdc-9239-b0f6affe2a32 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:21.389132 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:21.405985 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 48/93] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 13:59:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 13:59:21.406871 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544a160b-0724-45c0-bcab-212e5eeafe0a None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:21.516365 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544a160b-0724-45c0-bcab-212e5eeafe0a None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:21.564409 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c164c8-2e26-403a-a8d0-d89501c75a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:21.565022 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c164c8-2e26-403a-a8d0-d89501c75a69 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:21.565296 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c164c8-2e26-403a-a8d0-d89501c75a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:21.565566 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c164c8-2e26-403a-a8d0-d89501c75a69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:21.566294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c164c8-2e26-403a-a8d0-d89501c75a69 None admin] RBAC: Adding query filter params (role.id=e14564f00daf48a28d4e32da8cd2efad, scope.project.id=e10c65c7a17c44a7a8711e7701efdcd0, user.id=e46b19c26ed14ce08997d6b7fd3a2efb) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:21.566597 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c164c8-2e26-403a-a8d0-d89501c75a69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:21.568070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c164c8-2e26-403a-a8d0-d89501c75a69 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:21.579977 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 49/94] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 13:59:21 2022] GET /identity/v3/role_assignments?user.id=e46b19c26ed14ce08997d6b7fd3a2efb&scope.project.id=e10c65c7a17c44a7a8711e7701efdcd0&role.id=e14564f00daf48a28d4e32da8cd2efad => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:21.580827 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c164c8-2e26-403a-a8d0-d89501c75a69 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:21.628358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:21.631988 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0351fa5-d53a-4bdc-9239-b0f6affe2a32 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:21.742997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c164c8-2e26-403a-a8d0-d89501c75a69 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:21.771774 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec9359-b83f-4770-b760-d7ea22314e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:21.772244 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec9359-b83f-4770-b760-d7ea22314e4b None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:21.772530 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec9359-b83f-4770-b760-d7ea22314e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:21.772806 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec9359-b83f-4770-b760-d7ea22314e4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:21.773690 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec9359-b83f-4770-b760-d7ea22314e4b None admin] RBAC: Adding query filter params (role.id=bcde1b76af8048deb2829c5aaa27296c, scope.project.id=e10c65c7a17c44a7a8711e7701efdcd0, user.id=96b89bc850f04d96bedaea747c2c46b7) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:21.773864 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec9359-b83f-4770-b760-d7ea22314e4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:21.775366 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec9359-b83f-4770-b760-d7ea22314e4b None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:21.782654 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67478a9a-9ffb-4153-9d79-e2ada2b3fd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:21.783699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67478a9a-9ffb-4153-9d79-e2ada2b3fd15 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:21.784174 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67478a9a-9ffb-4153-9d79-e2ada2b3fd15 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:21.784639 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67478a9a-9ffb-4153-9d79-e2ada2b3fd15 None admin] PATH_INFO: `/v3/projects/e1b78198b2b24bfd8f607348b26438ad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:21.786951 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67478a9a-9ffb-4153-9d79-e2ada2b3fd15 None admin] RBAC: Authorizing `identity:get_project(project_id=e1b78198b2b24bfd8f607348b26438ad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:21.788821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67478a9a-9ffb-4153-9d79-e2ada2b3fd15 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:21.790881 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 50/95] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 13:59:21 2022] GET /identity/v3/projects/e1b78198b2b24bfd8f607348b26438ad => generated 312 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:21.797630 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67478a9a-9ffb-4153-9d79-e2ada2b3fd15 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:21.808698 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 46/96] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 13:59:21 2022] GET /identity/v3/role_assignments?user.id=96b89bc850f04d96bedaea747c2c46b7&scope.project.id=e10c65c7a17c44a7a8711e7701efdcd0&role.id=bcde1b76af8048deb2829c5aaa27296c => generated 259 bytes in 440 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:21.951693 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67478a9a-9ffb-4153-9d79-e2ada2b3fd15 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:21.963236 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:22.003667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c094bd-dbd4-4684-b8e6-032f41262886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:22.004662 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c094bd-dbd4-4684-b8e6-032f41262886 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:22.005137 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c094bd-dbd4-4684-b8e6-032f41262886 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:22.005619 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c094bd-dbd4-4684-b8e6-032f41262886 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:22.006687 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c094bd-dbd4-4684-b8e6-032f41262886 None admin] RBAC: Adding query filter params (role.id=d0a6f7c5c34a4925b9efb7dd4ac915fc, scope.project.id=e1b78198b2b24bfd8f607348b26438ad, user.id=96b89bc850f04d96bedaea747c2c46b7) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:22.007243 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c094bd-dbd4-4684-b8e6-032f41262886 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:22.008924 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c094bd-dbd4-4684-b8e6-032f41262886 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:22.015546 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 51/97] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 13:59:21 2022] GET /identity/v3/role_assignments?user.id=96b89bc850f04d96bedaea747c2c46b7&scope.project.id=e1b78198b2b24bfd8f607348b26438ad&role.id=d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:23.999645 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56e0a1-627f-4d3a-946a-b23e8703bc7b None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:24.000145 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56e0a1-627f-4d3a-946a-b23e8703bc7b None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:24.000215 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56e0a1-627f-4d3a-946a-b23e8703bc7b None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:24.001546 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 47/98] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:24.006748 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d1257a-aae9-40f9-846c-0a983705938f None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:24.007572 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d1257a-aae9-40f9-846c-0a983705938f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:24.008040 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d1257a-aae9-40f9-846c-0a983705938f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:24.036019 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-76d1257a-aae9-40f9-846c-0a983705938f None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:24.037670 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-76d1257a-aae9-40f9-846c-0a983705938f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:24.128209 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 52/99] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:24.222353 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3417e92-e294-48c2-90bd-2dd2a6be7813 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:24.222644 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3417e92-e294-48c2-90bd-2dd2a6be7813 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:24.222950 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3417e92-e294-48c2-90bd-2dd2a6be7813 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:24.224314 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 48/100] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:24.236196 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f368303-4138-4cc7-bd66-2f13f65a0245 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:24.236715 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f368303-4138-4cc7-bd66-2f13f65a0245 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:24.237166 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f368303-4138-4cc7-bd66-2f13f65a0245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:24.258999 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-4f368303-4138-4cc7-bd66-2f13f65a0245 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:24.260448 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-4f368303-4138-4cc7-bd66-2f13f65a0245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:24.353438 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 53/101] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 13:59:24.642709 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3417e92-e294-48c2-90bd-2dd2a6be7813 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:24.693380 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf8440-1211-4fac-bc6d-6dc43d19083f None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:24.693694 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf8440-1211-4fac-bc6d-6dc43d19083f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:24.693941 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf8440-1211-4fac-bc6d-6dc43d19083f None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:24.695645 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 54/102] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:24.701368 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb1909-6ff2-4d1c-934d-83624b87543e None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:24.701661 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb1909-6ff2-4d1c-934d-83624b87543e None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:24.701941 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb1909-6ff2-4d1c-934d-83624b87543e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:24.731397 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-dbbb1909-6ff2-4d1c-934d-83624b87543e None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:24.732720 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-dbbb1909-6ff2-4d1c-934d-83624b87543e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:24.751958 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:24.762873 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:24.815697 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3417e92-e294-48c2-90bd-2dd2a6be7813 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:24.844888 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 55/103] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:24.846639 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff1a15-5cce-4dd8-904e-d22c56100849 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:24.847165 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff1a15-5cce-4dd8-904e-d22c56100849 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:24.847673 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff1a15-5cce-4dd8-904e-d22c56100849 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:24.849990 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 56/104] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:24.850974 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff1a15-5cce-4dd8-904e-d22c56100849 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:24.878114 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2586395f-c0c6-4322-9f68-f716b21d4681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:24.878758 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586395f-c0c6-4322-9f68-f716b21d4681 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:24.879071 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586395f-c0c6-4322-9f68-f716b21d4681 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:24.879345 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586395f-c0c6-4322-9f68-f716b21d4681 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:24.904359 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2586395f-c0c6-4322-9f68-f716b21d4681 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:24.909192 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-2586395f-c0c6-4322-9f68-f716b21d4681 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 13:59:24.909192 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:24.909192 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:24.909192 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:24.909192 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:24.909192 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:24.909192 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:24.909192 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:24.909192 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:24.909192 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:24.909192 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:24.909192 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:24.909192 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:24.909192 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:24.909192 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:24.909192 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:24.910040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:24.910040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:24.910040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:24.910040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:24.910040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:24.910040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:24.910040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:24.910040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:24.910040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:24.910040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:24.910040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:24.910040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:24.910040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:24.910040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:24.910040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:24.910040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:24.911593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:24.911593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:24.911593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:24.911593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:24.911593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:24.911593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:24.911593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:24.911593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:24.911593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:24.911593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:24.911593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 13:59:24.911593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 13:59:24.911593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 49/105] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 13:59:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 13:59:24.919601 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56949b43-bead-424f-a693-464919291f19 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:24.919888 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56949b43-bead-424f-a693-464919291f19 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:24.920168 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56949b43-bead-424f-a693-464919291f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:24.959249 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-56949b43-bead-424f-a693-464919291f19 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:24.960422 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-56949b43-bead-424f-a693-464919291f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:24.984670 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff1a15-5cce-4dd8-904e-d22c56100849 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:25.021877 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897599f6-4313-41fc-ba62-167a81da3dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:25.023197 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897599f6-4313-41fc-ba62-167a81da3dd1 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:25.023673 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897599f6-4313-41fc-ba62-167a81da3dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:25.024111 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897599f6-4313-41fc-ba62-167a81da3dd1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:25.025375 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897599f6-4313-41fc-ba62-167a81da3dd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:25.027540 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897599f6-4313-41fc-ba62-167a81da3dd1 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:25.029285 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 57/106] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 13:59:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:25.030174 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897599f6-4313-41fc-ba62-167a81da3dd1 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:25.051456 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:25.149852 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897599f6-4313-41fc-ba62-167a81da3dd1 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:25.177720 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 50/107] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:25.179274 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56949b43-bead-424f-a693-464919291f19 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:25.206356 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc62de-5d85-438b-ae01-06e0f673429b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:25.207270 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc62de-5d85-438b-ae01-06e0f673429b None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:25.207686 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc62de-5d85-438b-ae01-06e0f673429b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:25.208103 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc62de-5d85-438b-ae01-06e0f673429b None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:25.208774 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cc62de-5d85-438b-ae01-06e0f673429b None admin] RBAC: Adding query filter params (name=member) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:25.209075 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cc62de-5d85-438b-ae01-06e0f673429b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:25.210421 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cc62de-5d85-438b-ae01-06e0f673429b None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:25.220688 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 58/108] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 12 13:59:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:25.221513 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc62de-5d85-438b-ae01-06e0f673429b None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:25.260350 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:25.302572 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56949b43-bead-424f-a693-464919291f19 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:25.309438 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:25.333970 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc62de-5d85-438b-ae01-06e0f673429b None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:25.337596 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26290e-1018-4717-9424-c0b4b807bd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:25.338208 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26290e-1018-4717-9424-c0b4b807bd09 None admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:25.338490 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26290e-1018-4717-9424-c0b4b807bd09 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:25.339353 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26290e-1018-4717-9424-c0b4b807bd09 None admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:25.340005 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df26290e-1018-4717-9424-c0b4b807bd09 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:25.341459 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df26290e-1018-4717-9424-c0b4b807bd09 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:25.368531 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 51/109] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Sep 12 13:59:25 2022] POST /identity/v3/projects => generated 302 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 13:59:25.369470 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39ad2d-dffe-4278-b9fc-023a067f5635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:25.370169 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39ad2d-dffe-4278-b9fc-023a067f5635 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:25.370449 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39ad2d-dffe-4278-b9fc-023a067f5635 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:25.370722 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39ad2d-dffe-4278-b9fc-023a067f5635 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:25.380701 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b80e6e-6067-499b-803e-9e713a0934c4 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:25.381018 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b80e6e-6067-499b-803e-9e713a0934c4 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:25.381267 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b80e6e-6067-499b-803e-9e713a0934c4 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:25.382320 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef39ad2d-dffe-4278-b9fc-023a067f5635 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:25.387148 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 52/110] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:25.387148 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b80e6e-6067-499b-803e-9e713a0934c4 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:25.388900 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-ef39ad2d-dffe-4278-b9fc-023a067f5635 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 13:59:25.388900 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:25.388900 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:25.388900 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:25.388900 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:25.388900 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:25.388900 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:25.388900 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:25.388900 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:25.388900 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:25.388900 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:25.388900 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:25.388900 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:25.388900 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:25.388900 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:25.388900 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:25.389737 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:25.389737 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:25.389737 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:25.389737 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:25.389737 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:25.389737 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:25.389737 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:25.389737 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:25.389737 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:25.389737 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:25.389737 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:25.389737 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:25.389737 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:25.389737 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:25.389737 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:25.389737 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:25.390489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:25.390489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:25.390489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:25.390489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:25.390489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:25.390489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:25.390489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:25.390489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:25.390489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:25.390489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:25.390489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 13:59:25.390489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 13:59:25.390489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 59/111] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 12 13:59:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 13:59:25.399538 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce725004-a0de-4c70-a985-17c6750699f2 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:25.399538 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce725004-a0de-4c70-a985-17c6750699f2 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:25.399538 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce725004-a0de-4c70-a985-17c6750699f2 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:25.399538 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 60/112] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:25.399538 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35e49d-f0ac-43d7-8251-564fb8e337ce None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:25.399538 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35e49d-f0ac-43d7-8251-564fb8e337ce None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:25.399995 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35e49d-f0ac-43d7-8251-564fb8e337ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:25.424331 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-7e35e49d-f0ac-43d7-8251-564fb8e337ce None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:25.425666 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7e35e49d-f0ac-43d7-8251-564fb8e337ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:25.504037 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 61/113] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:25.505137 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35e49d-f0ac-43d7-8251-564fb8e337ce None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:25.516050 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b80e6e-6067-499b-803e-9e713a0934c4 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:25.533307 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:25.560884 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034c943-1462-4cc2-b610-c73eea714e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:25.561962 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034c943-1462-4cc2-b610-c73eea714e34 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:25.562104 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034c943-1462-4cc2-b610-c73eea714e34 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:25.562409 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034c943-1462-4cc2-b610-c73eea714e34 None admin] PATH_INFO: `/v3/users/96b89bc850f04d96bedaea747c2c46b7` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:25.563904 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c034c943-1462-4cc2-b610-c73eea714e34 None admin] RBAC: Authorizing `identity:get_user(user_id=96b89bc850f04d96bedaea747c2c46b7)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:25.565475 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c034c943-1462-4cc2-b610-c73eea714e34 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:25.567257 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 53/114] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 13:59:25 2022] GET /identity/v3/users/96b89bc850f04d96bedaea747c2c46b7 => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:25.572101 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929cea0-c624-4ad8-9fbc-4a279ef08f1e None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:25.572101 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929cea0-c624-4ad8-9fbc-4a279ef08f1e None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:25.572101 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929cea0-c624-4ad8-9fbc-4a279ef08f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:25.593026 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-7929cea0-c624-4ad8-9fbc-4a279ef08f1e None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:25.594512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-7929cea0-c624-4ad8-9fbc-4a279ef08f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:25.609237 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:25.654714 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35e49d-f0ac-43d7-8251-564fb8e337ce None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:25.684091 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 54/115] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:25.684958 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7929cea0-c624-4ad8-9fbc-4a279ef08f1e None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:25.708750 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82835a7-172f-4aa0-80a5-a079e110ff90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:25.709373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82835a7-172f-4aa0-80a5-a079e110ff90 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:25.709657 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82835a7-172f-4aa0-80a5-a079e110ff90 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:25.709934 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82835a7-172f-4aa0-80a5-a079e110ff90 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:25.710614 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82835a7-172f-4aa0-80a5-a079e110ff90 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:25.710991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82835a7-172f-4aa0-80a5-a079e110ff90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:25.712443 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82835a7-172f-4aa0-80a5-a079e110ff90 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:25.732212 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 62/116] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 13:59:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:25.733040 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82835a7-172f-4aa0-80a5-a079e110ff90 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:25.804006 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7929cea0-c624-4ad8-9fbc-4a279ef08f1e None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:25.827132 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:25.855060 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838576a2-691f-46ad-b410-50334f5a373c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:25.856020 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838576a2-691f-46ad-b410-50334f5a373c None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:25.856462 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838576a2-691f-46ad-b410-50334f5a373c None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:25.856918 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838576a2-691f-46ad-b410-50334f5a373c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:25.867816 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838576a2-691f-46ad-b410-50334f5a373c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:25.872377 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82835a7-172f-4aa0-80a5-a079e110ff90 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:25.873418 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-838576a2-691f-46ad-b410-50334f5a373c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 13:59:25.873418 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:25.873418 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:25.873418 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:25.873418 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:25.873418 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:25.873418 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:25.873418 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:25.873418 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:25.873418 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:25.873418 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:25.873418 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:25.873418 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:25.873418 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:25.873418 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:25.873418 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:25.874253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:25.874253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:25.874253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:25.874253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:25.874253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:25.874253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:25.874253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:25.874253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:25.874253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:25.874253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:25.874253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:25.874253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:25.874253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:25.874253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:25.874253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:25.874253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:25.875112 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:25.875112 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:25.875112 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:25.875112 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:25.875112 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:25.875112 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:25.875112 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:25.875112 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:25.875112 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:25.875112 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:25.875112 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 13:59:25.875112 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 13:59:25.877097 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 55/117] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 13:59:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 13:59:25.882758 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838576a2-691f-46ad-b410-50334f5a373c None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:25.904380 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:25.910990 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0405ad-a23e-4c86-9d30-1d97f688c3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:25.911629 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0405ad-a23e-4c86-9d30-1d97f688c3a4 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:25.911906 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0405ad-a23e-4c86-9d30-1d97f688c3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:25.912192 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0405ad-a23e-4c86-9d30-1d97f688c3a4 None admin] PATH_INFO: `/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:25.913774 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0405ad-a23e-4c86-9d30-1d97f688c3a4 None admin] RBAC: Authorizing `identity:get_project(project_id=e10c65c7a17c44a7a8711e7701efdcd0)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:25.915496 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0405ad-a23e-4c86-9d30-1d97f688c3a4 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:25.918492 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 63/118] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 13:59:25 2022] GET /identity/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:25.919285 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0405ad-a23e-4c86-9d30-1d97f688c3a4 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:26.007306 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838576a2-691f-46ad-b410-50334f5a373c None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:26.038487 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0405ad-a23e-4c86-9d30-1d97f688c3a4 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:26.044589 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c0f0d-61e6-4804-ae5e-7aa38248b737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:26.045224 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c0f0d-61e6-4804-ae5e-7aa38248b737 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:26.045500 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c0f0d-61e6-4804-ae5e-7aa38248b737 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:26.045770 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c0f0d-61e6-4804-ae5e-7aa38248b737 None admin] PATH_INFO: `/v3/users/e46b19c26ed14ce08997d6b7fd3a2efb` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:26.046920 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c0f0d-61e6-4804-ae5e-7aa38248b737 None admin] RBAC: Authorizing `identity:get_user(user_id=e46b19c26ed14ce08997d6b7fd3a2efb)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:26.048368 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c0f0d-61e6-4804-ae5e-7aa38248b737 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:26.049924 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 56/119] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 13:59:25 2022] GET /identity/v3/users/e46b19c26ed14ce08997d6b7fd3a2efb => generated 247 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:26.055111 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c0f0d-61e6-4804-ae5e-7aa38248b737 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:26.074496 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5fb940-0d60-466c-9f22-cb080ad6fa35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:26.075036 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5fb940-0d60-466c-9f22-cb080ad6fa35 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:26.075317 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5fb940-0d60-466c-9f22-cb080ad6fa35 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:26.075590 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5fb940-0d60-466c-9f22-cb080ad6fa35 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:26.087313 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5fb940-0d60-466c-9f22-cb080ad6fa35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:26.092693 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-2d5fb940-0d60-466c-9f22-cb080ad6fa35 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 13:59:26.092693 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:26.092693 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:26.092693 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:26.092693 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:26.092693 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:26.092693 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:26.092693 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:26.092693 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:26.092693 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:26.092693 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:26.092693 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:26.092693 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:26.092693 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:26.092693 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:26.092693 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:26.093600 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:26.093600 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:26.093600 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:26.093600 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:26.093600 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:26.093600 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:26.093600 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:26.093600 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:26.093600 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:26.093600 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:26.093600 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:26.093600 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:26.093600 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:26.093600 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:26.093600 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:26.093600 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:26.094358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:26.094358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:26.094358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:26.094358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:26.094358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:26.094358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:26.094358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:26.094358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:26.094358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:26.094358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:26.094358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 13:59:26.094358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 13:59:26.098662 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 64/120] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 12 13:59:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 13:59:26.099575 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5fb940-0d60-466c-9f22-cb080ad6fa35 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:26.100102 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:26.160755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c0f0d-61e6-4804-ae5e-7aa38248b737 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:26.161591 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:26.197145 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c674b-be4b-44a1-93ca-0618624cdfad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:26.197795 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c674b-be4b-44a1-93ca-0618624cdfad None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:26.198099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c674b-be4b-44a1-93ca-0618624cdfad None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:26.198380 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c674b-be4b-44a1-93ca-0618624cdfad None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:26.199093 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5c674b-be4b-44a1-93ca-0618624cdfad None admin] RBAC: Adding query filter params (name=reader) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:26.199383 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5c674b-be4b-44a1-93ca-0618624cdfad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:26.200774 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5c674b-be4b-44a1-93ca-0618624cdfad None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:26.208100 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5fb940-0d60-466c-9f22-cb080ad6fa35 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:26.208822 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 57/121] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 12 13:59:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:26.209624 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c674b-be4b-44a1-93ca-0618624cdfad None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:26.249067 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab50fef-c5f5-41d4-97d4-eb9c18fd40a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:26.250034 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab50fef-c5f5-41d4-97d4-eb9c18fd40a0 None admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:26.250481 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab50fef-c5f5-41d4-97d4-eb9c18fd40a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:26.250953 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab50fef-c5f5-41d4-97d4-eb9c18fd40a0 None admin] PATH_INFO: `/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0/users/96b89bc850f04d96bedaea747c2c46b7/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:26.253130 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab50fef-c5f5-41d4-97d4-eb9c18fd40a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=e10c65c7a17c44a7a8711e7701efdcd0, user_id=96b89bc850f04d96bedaea747c2c46b7, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:26.254878 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab50fef-c5f5-41d4-97d4-eb9c18fd40a0 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:26.270535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 65/122] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 12 13:59:26 2022] PUT /identity/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0/users/96b89bc850f04d96bedaea747c2c46b7/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 13:59:26.271787 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab50fef-c5f5-41d4-97d4-eb9c18fd40a0 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:26.333420 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c674b-be4b-44a1-93ca-0618624cdfad None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:26.337517 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:26.385281 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30b16c-8b4b-46a0-816a-3508a522be48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:26.386215 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30b16c-8b4b-46a0-816a-3508a522be48 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:26.386592 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30b16c-8b4b-46a0-816a-3508a522be48 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:26.387006 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30b16c-8b4b-46a0-816a-3508a522be48 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:26.402169 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a30b16c-8b4b-46a0-816a-3508a522be48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:26.409527 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-1a30b16c-8b4b-46a0-816a-3508a522be48 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 13:59:26.409527 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:26.409527 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:26.409527 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:26.409527 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:26.409527 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:26.409527 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:26.409527 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:26.409527 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:26.409527 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:26.409527 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:26.409527 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:26.409527 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:26.409527 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:26.409527 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:26.409527 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:26.410480 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:26.410480 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:26.410480 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:26.410480 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:26.410480 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:26.410480 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:26.410480 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:26.410480 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:26.410480 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:26.410480 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:26.410480 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:26.410480 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:26.410480 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:26.410480 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:26.410480 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:26.410480 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:26.411291 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:26.411291 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:26.411291 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:26.411291 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:26.411291 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:26.411291 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:26.411291 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:26.411291 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:26.411291 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:26.411291 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:26.411291 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 13:59:26.411291 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 13:59:26.411291 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 58/123] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 13:59:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 13:59:26.411953 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30b16c-8b4b-46a0-816a-3508a522be48 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:26.418685 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab50fef-c5f5-41d4-97d4-eb9c18fd40a0 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:26.439793 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:26.464151 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8284a195-491f-461e-b463-012e1c8bf032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:26.465326 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284a195-491f-461e-b463-012e1c8bf032 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:26.465837 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284a195-491f-461e-b463-012e1c8bf032 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:26.468505 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284a195-491f-461e-b463-012e1c8bf032 None admin] PATH_INFO: `/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:26.470258 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8284a195-491f-461e-b463-012e1c8bf032 None admin] RBAC: Authorizing `identity:get_project(project_id=e10c65c7a17c44a7a8711e7701efdcd0)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:26.472176 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8284a195-491f-461e-b463-012e1c8bf032 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:26.474736 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 66/124] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 13:59:26 2022] GET /identity/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:26.476870 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8284a195-491f-461e-b463-012e1c8bf032 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:26.543377 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30b16c-8b4b-46a0-816a-3508a522be48 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:26.589156 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d93d920-bf7f-4b84-9e91-e0a28b3f909c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:26.590348 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93d920-bf7f-4b84-9e91-e0a28b3f909c None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:26.591257 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93d920-bf7f-4b84-9e91-e0a28b3f909c None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:26.592247 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93d920-bf7f-4b84-9e91-e0a28b3f909c None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:26.593378 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d93d920-bf7f-4b84-9e91-e0a28b3f909c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:26.594287 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d93d920-bf7f-4b84-9e91-e0a28b3f909c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:26.596768 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8284a195-491f-461e-b463-012e1c8bf032 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:26.598870 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d93d920-bf7f-4b84-9e91-e0a28b3f909c None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:26.610789 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 59/125] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 13:59:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:26.611952 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d93d920-bf7f-4b84-9e91-e0a28b3f909c None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:26.640584 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd726a69-96a3-441c-9458-7c4df3e0e578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:26.641274 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd726a69-96a3-441c-9458-7c4df3e0e578 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:26.641559 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd726a69-96a3-441c-9458-7c4df3e0e578 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:26.641887 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd726a69-96a3-441c-9458-7c4df3e0e578 None admin] PATH_INFO: `/v3/users/c007b65736674f0c980027d5463e5710` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:26.644941 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:26.661396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd726a69-96a3-441c-9458-7c4df3e0e578 None admin] RBAC: Authorizing `identity:get_user(user_id=c007b65736674f0c980027d5463e5710)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:26.662926 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd726a69-96a3-441c-9458-7c4df3e0e578 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:26.664892 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 67/126] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 13:59:26 2022] GET /identity/v3/users/c007b65736674f0c980027d5463e5710 => generated 289 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:26.670669 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd726a69-96a3-441c-9458-7c4df3e0e578 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:26.726042 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:26.755046 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d93d920-bf7f-4b84-9e91-e0a28b3f909c None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:26.785234 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd726a69-96a3-441c-9458-7c4df3e0e578 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:26.802478 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4ef11-a4a4-48d5-9ac0-ff5792dca7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:26.803811 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4ef11-a4a4-48d5-9ac0-ff5792dca7b5 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:26.804288 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4ef11-a4a4-48d5-9ac0-ff5792dca7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:26.804765 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4ef11-a4a4-48d5-9ac0-ff5792dca7b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:26.805614 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b4ef11-a4a4-48d5-9ac0-ff5792dca7b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:26.806631 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b4ef11-a4a4-48d5-9ac0-ff5792dca7b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:26.809371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b4ef11-a4a4-48d5-9ac0-ff5792dca7b5 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:26.820891 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b137e-fea9-44eb-94f3-83509bc3ce6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:26.821326 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 60/127] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 12 13:59:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:26.821624 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b137e-fea9-44eb-94f3-83509bc3ce6a None admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:26.822270 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b137e-fea9-44eb-94f3-83509bc3ce6a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:26.822554 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b137e-fea9-44eb-94f3-83509bc3ce6a None admin] PATH_INFO: `/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0/users/e46b19c26ed14ce08997d6b7fd3a2efb/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:26.824065 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4ef11-a4a4-48d5-9ac0-ff5792dca7b5 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:26.825167 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b137e-fea9-44eb-94f3-83509bc3ce6a None admin] RBAC: Authorizing `identity:create_grant(project_id=e10c65c7a17c44a7a8711e7701efdcd0, user_id=e46b19c26ed14ce08997d6b7fd3a2efb, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:26.826910 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b137e-fea9-44eb-94f3-83509bc3ce6a None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:26.845701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 68/128] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 12 13:59:26 2022] PUT /identity/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0/users/e46b19c26ed14ce08997d6b7fd3a2efb/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 13:59:26.852267 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b137e-fea9-44eb-94f3-83509bc3ce6a None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:26.927246 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:26.938066 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4ef11-a4a4-48d5-9ac0-ff5792dca7b5 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:26.976482 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c635fc5c-4b67-4917-81c4-c87f2be04c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:26.977301 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c635fc5c-4b67-4917-81c4-c87f2be04c5b None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:26.977688 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c635fc5c-4b67-4917-81c4-c87f2be04c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:26.978064 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c635fc5c-4b67-4917-81c4-c87f2be04c5b None admin] PATH_INFO: `/v3/users/96b89bc850f04d96bedaea747c2c46b7` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:26.979414 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c635fc5c-4b67-4917-81c4-c87f2be04c5b None admin] RBAC: Authorizing `identity:get_user(user_id=96b89bc850f04d96bedaea747c2c46b7)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:26.981147 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c635fc5c-4b67-4917-81c4-c87f2be04c5b None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:26.983038 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 61/129] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 13:59:26 2022] GET /identity/v3/users/96b89bc850f04d96bedaea747c2c46b7 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:26.984031 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c635fc5c-4b67-4917-81c4-c87f2be04c5b None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:27.093710 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b137e-fea9-44eb-94f3-83509bc3ce6a None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:27.096346 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:27.104104 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c635fc5c-4b67-4917-81c4-c87f2be04c5b None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:27.145489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a643f3c-75f0-4135-8ca6-0d331123718f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:27.146523 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a643f3c-75f0-4135-8ca6-0d331123718f None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:27.147053 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a643f3c-75f0-4135-8ca6-0d331123718f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:27.147535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a643f3c-75f0-4135-8ca6-0d331123718f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:27.159214 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8268832e-68c5-4791-b32b-9728ba3adbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:27.159866 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268832e-68c5-4791-b32b-9728ba3adbe7 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:27.160146 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268832e-68c5-4791-b32b-9728ba3adbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:27.160956 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268832e-68c5-4791-b32b-9728ba3adbe7 None admin] PATH_INFO: `/v3/users/96b89bc850f04d96bedaea747c2c46b7` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:27.161564 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8268832e-68c5-4791-b32b-9728ba3adbe7 None admin] RBAC: Authorizing `identity:get_user(user_id=96b89bc850f04d96bedaea747c2c46b7)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:27.163674 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8268832e-68c5-4791-b32b-9728ba3adbe7 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:27.165272 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 62/130] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 13:59:26 2022] GET /identity/v3/users/96b89bc850f04d96bedaea747c2c46b7 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:27.166101 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8268832e-68c5-4791-b32b-9728ba3adbe7 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:27.196464 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:27.231190 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a643f3c-75f0-4135-8ca6-0d331123718f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:27.233019 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a643f3c-75f0-4135-8ca6-0d331123718f None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:27.281065 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8268832e-68c5-4791-b32b-9728ba3adbe7 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:27.316283 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8536f2-74b6-423a-bbb2-7a9cf403cd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:27.316917 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8536f2-74b6-423a-bbb2-7a9cf403cd31 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:27.317200 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8536f2-74b6-423a-bbb2-7a9cf403cd31 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:27.317699 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8536f2-74b6-423a-bbb2-7a9cf403cd31 None admin] PATH_INFO: `/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:27.318664 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8536f2-74b6-423a-bbb2-7a9cf403cd31 None admin] RBAC: Authorizing `identity:get_project(project_id=e10c65c7a17c44a7a8711e7701efdcd0)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:27.320071 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8536f2-74b6-423a-bbb2-7a9cf403cd31 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:27.322925 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 63/131] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 13:59:27 2022] GET /identity/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:27.323872 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8536f2-74b6-423a-bbb2-7a9cf403cd31 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:27.353109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 69/132] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 13:59:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 13:59:27.354316 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a643f3c-75f0-4135-8ca6-0d331123718f None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:27.357806 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:27.460949 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a643f3c-75f0-4135-8ca6-0d331123718f None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:27.481692 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:27.502034 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262170e-bda9-4a5c-860a-87d9f59075e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:27.502596 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8536f2-74b6-423a-bbb2-7a9cf403cd31 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:27.502727 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262170e-bda9-4a5c-860a-87d9f59075e4 None admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:27.503021 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262170e-bda9-4a5c-860a-87d9f59075e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:27.503298 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262170e-bda9-4a5c-860a-87d9f59075e4 None admin] PATH_INFO: `/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0/users/96b89bc850f04d96bedaea747c2c46b7/roles/bcde1b76af8048deb2829c5aaa27296c` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:27.505547 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262170e-bda9-4a5c-860a-87d9f59075e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=e10c65c7a17c44a7a8711e7701efdcd0, user_id=96b89bc850f04d96bedaea747c2c46b7, role_id=bcde1b76af8048deb2829c5aaa27296c)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:27.507025 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262170e-bda9-4a5c-860a-87d9f59075e4 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:27.527101 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 70/133] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 12 13:59:27 2022] PUT /identity/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0/users/96b89bc850f04d96bedaea747c2c46b7/roles/bcde1b76af8048deb2829c5aaa27296c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 13:59:27.528092 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262170e-bda9-4a5c-860a-87d9f59075e4 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:27.603467 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ed683-6e3a-49d0-8255-6d16d8b8654c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:27.604178 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ed683-6e3a-49d0-8255-6d16d8b8654c None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:27.604476 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ed683-6e3a-49d0-8255-6d16d8b8654c None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:27.604755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ed683-6e3a-49d0-8255-6d16d8b8654c None admin] PATH_INFO: `/v3/projects/e1b78198b2b24bfd8f607348b26438ad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:27.608829 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ed683-6e3a-49d0-8255-6d16d8b8654c None admin] RBAC: Authorizing `identity:get_project(project_id=e1b78198b2b24bfd8f607348b26438ad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:27.610444 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ed683-6e3a-49d0-8255-6d16d8b8654c None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:27.612487 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 64/134] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 13:59:27 2022] GET /identity/v3/projects/e1b78198b2b24bfd8f607348b26438ad => generated 312 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:27.619668 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ed683-6e3a-49d0-8255-6d16d8b8654c None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:27.635227 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:27.706830 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262170e-bda9-4a5c-860a-87d9f59075e4 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:27.747228 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6acda81-e682-4947-82c7-d8f4c60354e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:27.748028 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acda81-e682-4947-82c7-d8f4c60354e2 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:27.748315 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acda81-e682-4947-82c7-d8f4c60354e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:27.748594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acda81-e682-4947-82c7-d8f4c60354e2 None admin] PATH_INFO: `/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:27.749829 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6acda81-e682-4947-82c7-d8f4c60354e2 None admin] RBAC: Authorizing `identity:get_project(project_id=e10c65c7a17c44a7a8711e7701efdcd0)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:27.751385 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6acda81-e682-4947-82c7-d8f4c60354e2 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:27.753273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 71/135] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 13:59:27 2022] GET /identity/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:27.759214 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6acda81-e682-4947-82c7-d8f4c60354e2 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:27.861598 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ed683-6e3a-49d0-8255-6d16d8b8654c None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:27.921613 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6acda81-e682-4947-82c7-d8f4c60354e2 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:27.927583 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42da9919-3f32-4475-aa47-aac4e1ae246b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:27.928488 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da9919-3f32-4475-aa47-aac4e1ae246b None admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:27.928988 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da9919-3f32-4475-aa47-aac4e1ae246b None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:27.929552 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da9919-3f32-4475-aa47-aac4e1ae246b None admin] PATH_INFO: `/v3/projects/e1b78198b2b24bfd8f607348b26438ad/users/96b89bc850f04d96bedaea747c2c46b7/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:27.932106 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42da9919-3f32-4475-aa47-aac4e1ae246b None admin] RBAC: Authorizing `identity:create_grant(project_id=e1b78198b2b24bfd8f607348b26438ad, user_id=96b89bc850f04d96bedaea747c2c46b7, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:27.934301 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42da9919-3f32-4475-aa47-aac4e1ae246b None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:27.951818 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 65/136] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 12 13:59:27 2022] PUT /identity/v3/projects/e1b78198b2b24bfd8f607348b26438ad/users/96b89bc850f04d96bedaea747c2c46b7/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 13:59:27.953704 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57973a50-36ba-45b4-96da-f3e7a71c8a4d None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:27.954216 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57973a50-36ba-45b4-96da-f3e7a71c8a4d None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:27.954688 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57973a50-36ba-45b4-96da-f3e7a71c8a4d None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:27.956809 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 66/137] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:27.973359 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c7532-fc47-4033-87d1-5e187ba893bd None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:27.974043 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c7532-fc47-4033-87d1-5e187ba893bd None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:27.974564 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c7532-fc47-4033-87d1-5e187ba893bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:28.004795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:28.006153 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-559c7532-fc47-4033-87d1-5e187ba893bd None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:28.008360 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-559c7532-fc47-4033-87d1-5e187ba893bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:28.038134 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:28.043573 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9c570-084f-4c73-8a7d-f17a8b70d189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:28.044202 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9c570-084f-4c73-8a7d-f17a8b70d189 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:28.044488 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9c570-084f-4c73-8a7d-f17a8b70d189 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:28.044757 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9c570-084f-4c73-8a7d-f17a8b70d189 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:28.045504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e9c570-084f-4c73-8a7d-f17a8b70d189 None admin] RBAC: Adding query filter params (role.id=dab439aae69348a183b07971883cebfb, scope.project.id=e10c65c7a17c44a7a8711e7701efdcd0, user.id=c007b65736674f0c980027d5463e5710) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:28.045810 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e9c570-084f-4c73-8a7d-f17a8b70d189 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:28.047295 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e9c570-084f-4c73-8a7d-f17a8b70d189 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:28.073652 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 72/138] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 13:59:27 2022] GET /identity/v3/role_assignments?user.id=c007b65736674f0c980027d5463e5710&scope.project.id=e10c65c7a17c44a7a8711e7701efdcd0&role.id=dab439aae69348a183b07971883cebfb => generated 259 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 13:59:28.137818 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 67/139] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 13:59:28.443822 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9c570-084f-4c73-8a7d-f17a8b70d189 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:28.566105 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9c570-084f-4c73-8a7d-f17a8b70d189 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:28.614210 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283dfbec-8e87-49f5-9ca0-b498c4a2117c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:28.615051 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283dfbec-8e87-49f5-9ca0-b498c4a2117c None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:28.615431 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283dfbec-8e87-49f5-9ca0-b498c4a2117c None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:28.615804 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283dfbec-8e87-49f5-9ca0-b498c4a2117c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:28.617042 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283dfbec-8e87-49f5-9ca0-b498c4a2117c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:28.618579 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283dfbec-8e87-49f5-9ca0-b498c4a2117c None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:28.620644 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 73/140] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 13:59:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:28.628249 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c7532-fc47-4033-87d1-5e187ba893bd None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:28.746900 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c7532-fc47-4033-87d1-5e187ba893bd None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:28.791661 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1456f8-6714-4859-a29d-02907a9dc240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:28.792974 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1456f8-6714-4859-a29d-02907a9dc240 None admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:28.793351 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1456f8-6714-4859-a29d-02907a9dc240 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:28.793736 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1456f8-6714-4859-a29d-02907a9dc240 None admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:28.794502 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1456f8-6714-4859-a29d-02907a9dc240 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:28.796069 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1456f8-6714-4859-a29d-02907a9dc240 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:28.826546 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 68/141] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Sep 12 13:59:28 2022] POST /identity/v3/users => generated 283 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 13:59:29.231498 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9200e4-636d-43f2-90a4-06c83da5959c None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:29.231983 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9200e4-636d-43f2-90a4-06c83da5959c None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:29.232429 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9200e4-636d-43f2-90a4-06c83da5959c None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:29.234043 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 74/142] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:29.239210 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ef58d-5be7-4579-9683-31ece763780a None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:29.239503 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ef58d-5be7-4579-9683-31ece763780a None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:29.239782 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ef58d-5be7-4579-9683-31ece763780a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:29.250544 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:29.267265 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-687ef58d-5be7-4579-9683-31ece763780a None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:29.268458 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-687ef58d-5be7-4579-9683-31ece763780a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:29.354490 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 69/143] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:29.808764 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9200e4-636d-43f2-90a4-06c83da5959c None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:29.890677 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:29.936052 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9200e4-636d-43f2-90a4-06c83da5959c None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:29.973200 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b5e0e-7948-4c00-83a1-8b4900bcad69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:29.973844 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b5e0e-7948-4c00-83a1-8b4900bcad69 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:29.974418 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b5e0e-7948-4c00-83a1-8b4900bcad69 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:29.974691 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b5e0e-7948-4c00-83a1-8b4900bcad69 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:29.987707 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b5e0e-7948-4c00-83a1-8b4900bcad69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:29.993282 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-f36b5e0e-7948-4c00-83a1-8b4900bcad69 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 13:59:29.993282 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:29.993282 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:29.993282 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:29.993282 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:29.993282 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:29.993282 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:29.993282 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:29.993282 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:29.993282 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:29.993282 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:29.993282 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:29.993282 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:29.993282 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:29.993282 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:29.993282 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:29.994163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:29.994163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:29.994163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:29.994163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:29.994163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:29.994163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:29.994163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:29.994163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:29.994163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:29.994163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:29.994163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:29.994163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:29.994163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:29.994163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:29.994163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:29.994163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:29.995111 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:29.995111 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:29.995111 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:29.995111 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:29.995111 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:29.995111 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:29.995111 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:29.995111 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:29.995111 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:29.995111 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:29.995111 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 13:59:29.995111 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 13:59:29.995111 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 75/144] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 13:59:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 13:59:30.002378 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ef58d-5be7-4579-9683-31ece763780a None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:30.113860 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ef58d-5be7-4579-9683-31ece763780a None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:30.152364 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c0872-8689-4614-b08d-332e7d031570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:30.152989 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c0872-8689-4614-b08d-332e7d031570 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:30.153268 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c0872-8689-4614-b08d-332e7d031570 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:30.153543 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c0872-8689-4614-b08d-332e7d031570 None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:30.154218 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741c0872-8689-4614-b08d-332e7d031570 None admin] RBAC: Adding query filter params (name=member) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:30.154516 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741c0872-8689-4614-b08d-332e7d031570 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:30.155930 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741c0872-8689-4614-b08d-332e7d031570 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:30.157661 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:30.172226 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 70/145] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 12 13:59:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:30.179801 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b5e0e-7948-4c00-83a1-8b4900bcad69 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:30.309049 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b5e0e-7948-4c00-83a1-8b4900bcad69 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:30.341220 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:30.353721 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a521b3-1911-4ea2-ab8a-d33caeabff1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:30.354399 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a521b3-1911-4ea2-ab8a-d33caeabff1f None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:30.355125 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a521b3-1911-4ea2-ab8a-d33caeabff1f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:30.355407 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a521b3-1911-4ea2-ab8a-d33caeabff1f None admin] PATH_INFO: `/v3/users/96b89bc850f04d96bedaea747c2c46b7` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:30.356560 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a521b3-1911-4ea2-ab8a-d33caeabff1f None admin] RBAC: Authorizing `identity:get_user(user_id=96b89bc850f04d96bedaea747c2c46b7)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:30.358001 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a521b3-1911-4ea2-ab8a-d33caeabff1f None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:30.360005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 76/146] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 13:59:30 2022] GET /identity/v3/users/96b89bc850f04d96bedaea747c2c46b7 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:30.366378 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c0872-8689-4614-b08d-332e7d031570 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:30.399611 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11691152-ed9a-44fb-a536-ba56874c29ca None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:30.399912 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11691152-ed9a-44fb-a536-ba56874c29ca None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:30.400234 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11691152-ed9a-44fb-a536-ba56874c29ca None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:30.402018 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 77/147] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:30.415418 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52376c-2c0d-4529-99f8-c190052aaa44 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:30.415705 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52376c-2c0d-4529-99f8-c190052aaa44 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:30.415972 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52376c-2c0d-4529-99f8-c190052aaa44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:30.439492 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-ec52376c-2c0d-4529-99f8-c190052aaa44 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:30.441041 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ec52376c-2c0d-4529-99f8-c190052aaa44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:30.531183 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 78/148] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:30.550716 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c0872-8689-4614-b08d-332e7d031570 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:30.605265 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23b055b-256e-4e50-b8b0-d7c07bc16bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:30.605894 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b055b-256e-4e50-b8b0-d7c07bc16bae None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:30.606174 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b055b-256e-4e50-b8b0-d7c07bc16bae None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:30.606435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b055b-256e-4e50-b8b0-d7c07bc16bae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:30.664345 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b35765-9219-47c5-89a5-4f16c3add46a None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:30.664636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b35765-9219-47c5-89a5-4f16c3add46a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:30.664806 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b35765-9219-47c5-89a5-4f16c3add46a None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:30.666323 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 79/149] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 13:59:30.672062 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89043a56-d0c0-4e92-82d3-8475c7dce1fb None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:30.672337 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89043a56-d0c0-4e92-82d3-8475c7dce1fb None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:30.672607 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89043a56-d0c0-4e92-82d3-8475c7dce1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:30.693938 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:30.705241 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23b055b-256e-4e50-b8b0-d7c07bc16bae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:30.706717 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23b055b-256e-4e50-b8b0-d7c07bc16bae None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:30.709431 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-89043a56-d0c0-4e92-82d3-8475c7dce1fb None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:30.711427 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-89043a56-d0c0-4e92-82d3-8475c7dce1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:30.846688 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 80/150] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:30.890029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 71/151] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 13:59:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 13:59:30.896016 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89043a56-d0c0-4e92-82d3-8475c7dce1fb None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:30.917144 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23b055b-256e-4e50-b8b0-d7c07bc16bae None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:30.973112 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:31.067436 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:31.069551 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89043a56-d0c0-4e92-82d3-8475c7dce1fb None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:31.076244 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23b055b-256e-4e50-b8b0-d7c07bc16bae None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:31.135648 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f5900-d674-4832-9994-cfbc74c86b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:31.136380 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f5900-d674-4832-9994-cfbc74c86b54 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:31.136650 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f5900-d674-4832-9994-cfbc74c86b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:31.136925 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f5900-d674-4832-9994-cfbc74c86b54 None admin] PATH_INFO: `/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:31.138334 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5f5900-d674-4832-9994-cfbc74c86b54 None admin] RBAC: Authorizing `identity:get_project(project_id=e10c65c7a17c44a7a8711e7701efdcd0)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:31.139810 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5f5900-d674-4832-9994-cfbc74c86b54 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:31.141450 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 81/152] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 13:59:30 2022] GET /identity/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0 => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 13:59:31.147131 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc4af7e-f4df-48da-8cf9-8ae2595262d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:31.147292 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f5900-d674-4832-9994-cfbc74c86b54 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:31.147776 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc4af7e-f4df-48da-8cf9-8ae2595262d4 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:31.148056 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc4af7e-f4df-48da-8cf9-8ae2595262d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:31.148326 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc4af7e-f4df-48da-8cf9-8ae2595262d4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:31.175931 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc4af7e-f4df-48da-8cf9-8ae2595262d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:31.180941 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-bcc4af7e-f4df-48da-8cf9-8ae2595262d4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 13:59:31.180941 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:31.180941 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:31.180941 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:31.180941 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:31.180941 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:31.180941 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:31.180941 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:31.180941 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:31.180941 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:31.180941 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:31.180941 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:31.180941 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:31.180941 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:31.180941 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:31.180941 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:31.181870 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:31.181870 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:31.181870 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:31.181870 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:31.181870 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:31.181870 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:31.181870 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:31.181870 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:31.181870 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:31.181870 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:31.181870 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:31.181870 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:31.181870 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:31.181870 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:31.181870 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:31.181870 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:31.182721 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:31.182721 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:31.182721 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:31.182721 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:31.182721 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:31.182721 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:31.182721 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:31.182721 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:31.182721 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:31.182721 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:31.182721 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 13:59:31.182721 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 13:59:31.182721 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 72/153] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 12 13:59:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 13:59:31.188690 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc4af7e-f4df-48da-8cf9-8ae2595262d4 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:31.332476 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f5900-d674-4832-9994-cfbc74c86b54 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:31.370615 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:31.388722 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc4af7e-f4df-48da-8cf9-8ae2595262d4 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:31.406868 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248dc7a-7266-44dd-aac6-751fce2ca1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:31.406868 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248dc7a-7266-44dd-aac6-751fce2ca1ec None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:31.406868 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248dc7a-7266-44dd-aac6-751fce2ca1ec None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:31.406868 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248dc7a-7266-44dd-aac6-751fce2ca1ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:31.407273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8248dc7a-7266-44dd-aac6-751fce2ca1ec None admin] RBAC: Adding query filter params (role.id=d0a6f7c5c34a4925b9efb7dd4ac915fc, scope.project.id=e10c65c7a17c44a7a8711e7701efdcd0, user.id=96b89bc850f04d96bedaea747c2c46b7) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:31.407273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8248dc7a-7266-44dd-aac6-751fce2ca1ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:31.408691 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8248dc7a-7266-44dd-aac6-751fce2ca1ec None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:31.414865 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 82/154] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 13:59:31 2022] GET /identity/v3/role_assignments?user.id=96b89bc850f04d96bedaea747c2c46b7&scope.project.id=e10c65c7a17c44a7a8711e7701efdcd0&role.id=d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 608 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 13:59:31.419889 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248dc7a-7266-44dd-aac6-751fce2ca1ec None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:31.445647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c66ed34-0355-4aa5-a912-92f4be6b8abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:31.446664 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66ed34-0355-4aa5-a912-92f4be6b8abd None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:31.447482 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66ed34-0355-4aa5-a912-92f4be6b8abd None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:31.448529 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66ed34-0355-4aa5-a912-92f4be6b8abd None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:31.449400 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c66ed34-0355-4aa5-a912-92f4be6b8abd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:31.449888 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c66ed34-0355-4aa5-a912-92f4be6b8abd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:31.451497 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c66ed34-0355-4aa5-a912-92f4be6b8abd None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:31.465740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 73/155] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 13:59:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 13:59:31.472303 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c66ed34-0355-4aa5-a912-92f4be6b8abd None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:31.507721 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:31.662172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248dc7a-7266-44dd-aac6-751fce2ca1ec None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:31.665134 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c66ed34-0355-4aa5-a912-92f4be6b8abd None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:31.725166 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a12ead-e834-471a-bdd3-e53c413645cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:31.726255 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a12ead-e834-471a-bdd3-e53c413645cd None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:31.726803 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a12ead-e834-471a-bdd3-e53c413645cd None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:31.727320 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a12ead-e834-471a-bdd3-e53c413645cd None admin] PATH_INFO: `/v3/users/96b89bc850f04d96bedaea747c2c46b7` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:31.728933 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a12ead-e834-471a-bdd3-e53c413645cd None admin] RBAC: Authorizing `identity:get_user(user_id=96b89bc850f04d96bedaea747c2c46b7)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:31.730443 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21367c1-1e70-4949-b31b-1a62ba58e621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:31.731208 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21367c1-1e70-4949-b31b-1a62ba58e621 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:31.731553 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21367c1-1e70-4949-b31b-1a62ba58e621 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:31.731896 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21367c1-1e70-4949-b31b-1a62ba58e621 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:31.732112 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a12ead-e834-471a-bdd3-e53c413645cd None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:31.739936 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 74/156] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 13:59:31 2022] GET /identity/v3/users/96b89bc850f04d96bedaea747c2c46b7 => generated 275 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 13:59:31.741519 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c4e7e-6ab6-4104-afe2-4c9ad662d5db None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:31.742043 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c4e7e-6ab6-4104-afe2-4c9ad662d5db None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:31.742663 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c4e7e-6ab6-4104-afe2-4c9ad662d5db None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:31.744106 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 75/157] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:31.750604 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00adc2-f4ec-45d6-bcc0-53e39fa694b5 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:31.751256 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00adc2-f4ec-45d6-bcc0-53e39fa694b5 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:31.751727 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00adc2-f4ec-45d6-bcc0-53e39fa694b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:31.766943 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21367c1-1e70-4949-b31b-1a62ba58e621 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:31.769565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-f21367c1-1e70-4949-b31b-1a62ba58e621 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 13:59:31.769565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:31.769565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:31.769565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:31.769565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:31.769565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:31.769565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:31.769565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:31.769565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:31.769565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:31.769565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:31.769565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:31.769565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:31.769565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:31.769565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:31.769565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:31.770500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:31.770500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:31.770500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:31.770500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:31.770500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:31.770500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:31.770500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:31.770500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:31.770500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:31.770500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:31.770500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:31.770500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:31.770500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:31.770500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:31.770500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:31.770500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:31.771301 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:31.771301 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:31.771301 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:31.771301 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:31.771301 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:31.771301 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:31.771301 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:31.771301 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:31.771301 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:31.771301 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:31.771301 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 13:59:31.771301 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 13:59:31.771301 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 83/158] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 12 13:59:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 13:59:31.772668 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21367c1-1e70-4949-b31b-1a62ba58e621 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:31.779920 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-9c00adc2-f4ec-45d6-bcc0-53e39fa694b5 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:31.786934 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-9c00adc2-f4ec-45d6-bcc0-53e39fa694b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:31.890002 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:31.908026 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 76/159] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 13:59:31.909024 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00adc2-f4ec-45d6-bcc0-53e39fa694b5 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:31.944022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:32.034651 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21367c1-1e70-4949-b31b-1a62ba58e621 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:32.106900 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00adc2-f4ec-45d6-bcc0-53e39fa694b5 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:32.116393 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ecf2b-2449-4c86-b69c-aab22f9d8590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:32.117280 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ecf2b-2449-4c86-b69c-aab22f9d8590 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:32.117280 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ecf2b-2449-4c86-b69c-aab22f9d8590 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:32.117737 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ecf2b-2449-4c86-b69c-aab22f9d8590 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:32.178518 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a0683-692e-46ec-af63-efab80bc24dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:32.179168 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a0683-692e-46ec-af63-efab80bc24dc None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:32.179444 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a0683-692e-46ec-af63-efab80bc24dc None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:32.179714 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a0683-692e-46ec-af63-efab80bc24dc None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:32.180394 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76a0683-692e-46ec-af63-efab80bc24dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:32.180689 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76a0683-692e-46ec-af63-efab80bc24dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:32.182192 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76a0683-692e-46ec-af63-efab80bc24dc None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:32.201827 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 77/160] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 13:59:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:32.203239 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b9e98-eccf-45b0-b032-6c1386aba3a8 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:32.203522 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b9e98-eccf-45b0-b032-6c1386aba3a8 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:32.203802 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b9e98-eccf-45b0-b032-6c1386aba3a8 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:32.205030 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 78/161] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:32.211329 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ded17-107d-4cdb-ab62-fe6ce922fcb6 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:32.211602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ded17-107d-4cdb-ab62-fe6ce922fcb6 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:32.211874 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ded17-107d-4cdb-ab62-fe6ce922fcb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:32.232851 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668ecf2b-2449-4c86-b69c-aab22f9d8590 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:32.234329 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668ecf2b-2449-4c86-b69c-aab22f9d8590 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:32.251834 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-129ded17-107d-4cdb-ab62-fe6ce922fcb6 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:32.253034 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-129ded17-107d-4cdb-ab62-fe6ce922fcb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:32.297312 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:32.391204 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 79/162] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:32.392094 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129ded17-107d-4cdb-ab62-fe6ce922fcb6 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:32.407199 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:32.410663 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 84/163] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 13:59:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 639 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 13:59:32.411731 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ecf2b-2449-4c86-b69c-aab22f9d8590 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:32.515350 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129ded17-107d-4cdb-ab62-fe6ce922fcb6 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:32.577968 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac1bca-0931-4357-ab19-630639a2840f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:32.578560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac1bca-0931-4357-ab19-630639a2840f None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:32.578861 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac1bca-0931-4357-ab19-630639a2840f None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:32.579143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac1bca-0931-4357-ab19-630639a2840f None admin] PATH_INFO: `/v3/users/e46b19c26ed14ce08997d6b7fd3a2efb` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:32.584057 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac1bca-0931-4357-ab19-630639a2840f None admin] RBAC: Authorizing `identity:get_user(user_id=e46b19c26ed14ce08997d6b7fd3a2efb)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:32.585830 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac1bca-0931-4357-ab19-630639a2840f None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:32.588215 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 80/164] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 13:59:32 2022] GET /identity/v3/users/e46b19c26ed14ce08997d6b7fd3a2efb => generated 247 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:32.589526 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac1bca-0931-4357-ab19-630639a2840f None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:32.634594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ecf2b-2449-4c86-b69c-aab22f9d8590 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:32.716840 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41a41d-4798-40ae-8eab-1591d05ec616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:32.717484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41a41d-4798-40ae-8eab-1591d05ec616 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:32.717759 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41a41d-4798-40ae-8eab-1591d05ec616 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:32.718166 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41a41d-4798-40ae-8eab-1591d05ec616 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:32.739191 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:32.739947 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41a41d-4798-40ae-8eab-1591d05ec616 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:32.747051 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-8d41a41d-4798-40ae-8eab-1591d05ec616 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 13:59:32.747051 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:32.747051 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:32.747051 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:32.747051 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:32.747051 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:32.747051 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:32.747051 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:32.747051 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:32.747051 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:32.747051 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:32.747051 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:32.747051 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:32.747051 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:32.747051 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:32.747051 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:32.747914 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:32.747914 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:32.747914 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:32.747914 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:32.747914 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:32.747914 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:32.747914 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:32.747914 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:32.747914 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:32.747914 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:32.747914 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:32.747914 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:32.747914 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:32.747914 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:32.747914 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:32.747914 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:32.748691 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:32.748691 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:32.748691 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:32.748691 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:32.748691 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:32.748691 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:32.748691 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:32.748691 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:32.748691 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:32.748691 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:32.748691 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 13:59:32.748691 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 13:59:32.748691 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 85/165] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 13:59:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 13:59:32.749534 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41a41d-4798-40ae-8eab-1591d05ec616 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:32.763704 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:32.786294 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac1bca-0931-4357-ab19-630639a2840f None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:32.862819 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9652d-cf4a-4626-8765-12a7358e7e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:32.863911 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9652d-cf4a-4626-8765-12a7358e7e0d None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:32.864426 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9652d-cf4a-4626-8765-12a7358e7e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:32.864881 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9652d-cf4a-4626-8765-12a7358e7e0d None admin] PATH_INFO: `/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:32.866187 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb9652d-cf4a-4626-8765-12a7358e7e0d None admin] RBAC: Authorizing `identity:get_project(project_id=e10c65c7a17c44a7a8711e7701efdcd0)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:32.868028 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb9652d-cf4a-4626-8765-12a7358e7e0d None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:32.870303 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 81/166] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 13:59:32 2022] GET /identity/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0 => generated 298 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:32.877125 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9652d-cf4a-4626-8765-12a7358e7e0d None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:32.894861 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41a41d-4798-40ae-8eab-1591d05ec616 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:32.934775 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab8938-d8e7-4569-b4c0-ae04f9db25a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:32.935416 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab8938-d8e7-4569-b4c0-ae04f9db25a5 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:32.935697 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab8938-d8e7-4569-b4c0-ae04f9db25a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:32.935966 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab8938-d8e7-4569-b4c0-ae04f9db25a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:33.032526 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9652d-cf4a-4626-8765-12a7358e7e0d None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:33.053489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab8938-d8e7-4569-b4c0-ae04f9db25a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:33.055364 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab8938-d8e7-4569-b4c0-ae04f9db25a5 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:33.064843 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:33.097403 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab2f79f-b3d6-48ce-9c02-92a56a11d2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:33.097930 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2f79f-b3d6-48ce-9c02-92a56a11d2ec None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:33.098197 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2f79f-b3d6-48ce-9c02-92a56a11d2ec None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:33.098462 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2f79f-b3d6-48ce-9c02-92a56a11d2ec None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:33.099148 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab2f79f-b3d6-48ce-9c02-92a56a11d2ec None admin] RBAC: Adding query filter params (name=member) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:33.099444 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab2f79f-b3d6-48ce-9c02-92a56a11d2ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:33.100797 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab2f79f-b3d6-48ce-9c02-92a56a11d2ec None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:33.115936 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 82/167] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 12 13:59:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:33.116768 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab2f79f-b3d6-48ce-9c02-92a56a11d2ec None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:33.142448 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:33.257642 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 86/168] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 13:59:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 13:59:33.259417 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab8938-d8e7-4569-b4c0-ae04f9db25a5 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:33.292491 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab2f79f-b3d6-48ce-9c02-92a56a11d2ec None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:33.332335 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1216eb00-74e2-45c6-b4ea-8db21dd254ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:33.332989 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216eb00-74e2-45c6-b4ea-8db21dd254ba None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:33.333271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216eb00-74e2-45c6-b4ea-8db21dd254ba None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:33.333541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216eb00-74e2-45c6-b4ea-8db21dd254ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:33.334255 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1216eb00-74e2-45c6-b4ea-8db21dd254ba None admin] RBAC: Adding query filter params (role.id=bcde1b76af8048deb2829c5aaa27296c, scope.project.id=e10c65c7a17c44a7a8711e7701efdcd0, user.id=96b89bc850f04d96bedaea747c2c46b7) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:33.334555 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1216eb00-74e2-45c6-b4ea-8db21dd254ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:33.336341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1216eb00-74e2-45c6-b4ea-8db21dd254ba None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:33.356292 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 83/169] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 13:59:33 2022] GET /identity/v3/role_assignments?user.id=96b89bc850f04d96bedaea747c2c46b7&scope.project.id=e10c65c7a17c44a7a8711e7701efdcd0&role.id=bcde1b76af8048deb2829c5aaa27296c => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:33.357071 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1216eb00-74e2-45c6-b4ea-8db21dd254ba None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:33.410476 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab8938-d8e7-4569-b4c0-ae04f9db25a5 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:33.413524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:33.460093 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e5f2d-3cd9-4fc5-adc2-94fc78815128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:33.460744 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e5f2d-3cd9-4fc5-adc2-94fc78815128 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:33.461021 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e5f2d-3cd9-4fc5-adc2-94fc78815128 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:33.461294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e5f2d-3cd9-4fc5-adc2-94fc78815128 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:33.479445 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81e5f2d-3cd9-4fc5-adc2-94fc78815128 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:33.493781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-e81e5f2d-3cd9-4fc5-adc2-94fc78815128 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 13:59:33.493781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:33.493781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:33.493781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:33.493781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:33.493781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:33.493781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:33.493781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:33.493781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:33.493781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:33.493781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:33.493781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:33.493781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:33.493781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:33.493781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:33.493781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:33.495337 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:33.495337 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:33.495337 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:33.495337 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:33.495337 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:33.495337 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:33.495337 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:33.495337 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:33.495337 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:33.495337 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:33.495337 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:33.495337 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:33.495337 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:33.495337 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:33.495337 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:33.495337 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:33.496114 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:33.496114 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:33.496114 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:33.496114 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:33.496114 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:33.496114 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:33.496114 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:33.496114 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:33.496114 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:33.496114 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:33.496114 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 13:59:33.496114 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 13:59:33.496114 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 87/170] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 13:59:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 13:59:33.496763 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e5f2d-3cd9-4fc5-adc2-94fc78815128 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:33.508983 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:33.594405 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1216eb00-74e2-45c6-b4ea-8db21dd254ba None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:33.635801 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9464e04-58d1-4a28-96ff-86d2dd26d0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:33.636359 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9464e04-58d1-4a28-96ff-86d2dd26d0c1 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:33.636640 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9464e04-58d1-4a28-96ff-86d2dd26d0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:33.636916 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9464e04-58d1-4a28-96ff-86d2dd26d0c1 None admin] PATH_INFO: `/v3/users/96b89bc850f04d96bedaea747c2c46b7` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:33.638253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9464e04-58d1-4a28-96ff-86d2dd26d0c1 None admin] RBAC: Authorizing `identity:get_user(user_id=96b89bc850f04d96bedaea747c2c46b7)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:33.639747 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9464e04-58d1-4a28-96ff-86d2dd26d0c1 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:33.641352 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 84/171] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 13:59:33 2022] GET /identity/v3/users/96b89bc850f04d96bedaea747c2c46b7 => generated 275 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:33.646496 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9464e04-58d1-4a28-96ff-86d2dd26d0c1 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:33.647089 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e5f2d-3cd9-4fc5-adc2-94fc78815128 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:33.685670 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69216b2c-68e9-49be-bf8c-9b60b0fa0a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:33.687194 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69216b2c-68e9-49be-bf8c-9b60b0fa0a7c None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:33.687724 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69216b2c-68e9-49be-bf8c-9b60b0fa0a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:33.688186 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69216b2c-68e9-49be-bf8c-9b60b0fa0a7c None admin] PATH_INFO: `/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:33.689507 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69216b2c-68e9-49be-bf8c-9b60b0fa0a7c None admin] RBAC: Authorizing `identity:get_project(project_id=e10c65c7a17c44a7a8711e7701efdcd0)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:33.691174 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69216b2c-68e9-49be-bf8c-9b60b0fa0a7c None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:33.693021 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 88/172] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 13:59:33 2022] GET /identity/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:33.694042 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69216b2c-68e9-49be-bf8c-9b60b0fa0a7c None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:33.731258 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:33.801372 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9464e04-58d1-4a28-96ff-86d2dd26d0c1 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:33.839987 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69216b2c-68e9-49be-bf8c-9b60b0fa0a7c None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:33.858181 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401dc3cf-aa0c-415d-be8f-8683c54003fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:33.859360 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401dc3cf-aa0c-415d-be8f-8683c54003fb None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:33.859810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401dc3cf-aa0c-415d-be8f-8683c54003fb None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:33.860435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401dc3cf-aa0c-415d-be8f-8683c54003fb None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:33.861609 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401dc3cf-aa0c-415d-be8f-8683c54003fb None admin] RBAC: Adding query filter params (name=reader) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:33.862097 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401dc3cf-aa0c-415d-be8f-8683c54003fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:33.863669 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401dc3cf-aa0c-415d-be8f-8683c54003fb None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:33.870994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:33.878188 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 85/173] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 12 13:59:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:33.879448 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401dc3cf-aa0c-415d-be8f-8683c54003fb None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:33.888403 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaadc876-90e4-40e1-89df-bf057097875e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:33.888948 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaadc876-90e4-40e1-89df-bf057097875e None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:33.889226 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaadc876-90e4-40e1-89df-bf057097875e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:33.889494 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaadc876-90e4-40e1-89df-bf057097875e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:33.964287 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaadc876-90e4-40e1-89df-bf057097875e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:33.965745 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaadc876-90e4-40e1-89df-bf057097875e None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:33.994797 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401dc3cf-aa0c-415d-be8f-8683c54003fb None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:33.994964 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:34.028703 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4590b99-33d5-4e96-b078-245acfd25a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:34.029302 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4590b99-33d5-4e96-b078-245acfd25a33 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:34.029578 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4590b99-33d5-4e96-b078-245acfd25a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:34.029850 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4590b99-33d5-4e96-b078-245acfd25a33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:34.030570 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4590b99-33d5-4e96-b078-245acfd25a33 None admin] RBAC: Adding query filter params (role.id=e14564f00daf48a28d4e32da8cd2efad, scope.project.id=e10c65c7a17c44a7a8711e7701efdcd0, user.id=e46b19c26ed14ce08997d6b7fd3a2efb) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:34.030906 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4590b99-33d5-4e96-b078-245acfd25a33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:34.032335 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4590b99-33d5-4e96-b078-245acfd25a33 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:34.037905 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 86/174] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 13:59:33 2022] GET /identity/v3/role_assignments?user.id=e46b19c26ed14ce08997d6b7fd3a2efb&scope.project.id=e10c65c7a17c44a7a8711e7701efdcd0&role.id=e14564f00daf48a28d4e32da8cd2efad => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:34.039198 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8186d888-2999-4521-b9d5-bb4ce0e06323 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:34.039478 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8186d888-2999-4521-b9d5-bb4ce0e06323 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:34.039750 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8186d888-2999-4521-b9d5-bb4ce0e06323 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:34.040965 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 87/175] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:34.046021 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8186d888-2999-4521-b9d5-bb4ce0e06323 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:34.077811 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 89/176] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 13:59:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 13:59:34.084124 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fdb6f6-bb33-430e-a915-ca2267f79d2b None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:34.084407 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fdb6f6-bb33-430e-a915-ca2267f79d2b None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:34.084678 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fdb6f6-bb33-430e-a915-ca2267f79d2b None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:34.091273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 90/177] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:34.091273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e860a-a83d-4cff-aea2-36486d0853b6 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:34.091273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e860a-a83d-4cff-aea2-36486d0853b6 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:34.091273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e860a-a83d-4cff-aea2-36486d0853b6 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:34.091273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 91/178] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:34.091273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31154c-bd54-4dcc-968c-ba7a17c555bf None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:34.091273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31154c-bd54-4dcc-968c-ba7a17c555bf None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:34.091797 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31154c-bd54-4dcc-968c-ba7a17c555bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:34.111227 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-cb31154c-bd54-4dcc-968c-ba7a17c555bf None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:34.112630 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-cb31154c-bd54-4dcc-968c-ba7a17c555bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:34.130332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:34.175065 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8186d888-2999-4521-b9d5-bb4ce0e06323 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:34.198863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 92/179] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:34.198863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb31154c-bd54-4dcc-968c-ba7a17c555bf None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:34.221206 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb73a41-d1c5-41f1-8328-aa71d4906914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:34.221837 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb73a41-d1c5-41f1-8328-aa71d4906914 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:34.222116 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb73a41-d1c5-41f1-8328-aa71d4906914 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:34.222382 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb73a41-d1c5-41f1-8328-aa71d4906914 None admin] PATH_INFO: `/v3/users/c007b65736674f0c980027d5463e5710` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:34.223577 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb73a41-d1c5-41f1-8328-aa71d4906914 None admin] RBAC: Authorizing `identity:get_user(user_id=c007b65736674f0c980027d5463e5710)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:34.225012 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb73a41-d1c5-41f1-8328-aa71d4906914 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:34.226584 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 88/180] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 13:59:34 2022] GET /identity/v3/users/c007b65736674f0c980027d5463e5710 => generated 289 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:34.227998 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4c495-9607-48c8-8953-a05b2a214372 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:34.228283 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4c495-9607-48c8-8953-a05b2a214372 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:34.228553 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4c495-9607-48c8-8953-a05b2a214372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:34.249447 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-31b4c495-9607-48c8-8953-a05b2a214372 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:34.251956 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-31b4c495-9607-48c8-8953-a05b2a214372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:34.277367 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:34.329560 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb31154c-bd54-4dcc-968c-ba7a17c555bf None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:34.336013 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 89/181] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:34.337066 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a863e56-ef40-4f2f-957c-d443e1a9810e None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:34.337541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a863e56-ef40-4f2f-957c-d443e1a9810e None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:34.337644 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a863e56-ef40-4f2f-957c-d443e1a9810e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:34.357631 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-9a863e56-ef40-4f2f-957c-d443e1a9810e None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:34.359489 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-9a863e56-ef40-4f2f-957c-d443e1a9810e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:34.367524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887e7c6-4dd3-474d-8708-ebb0a9d47fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:34.368217 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887e7c6-4dd3-474d-8708-ebb0a9d47fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:34.368496 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887e7c6-4dd3-474d-8708-ebb0a9d47fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:34.368808 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887e7c6-4dd3-474d-8708-ebb0a9d47fa3 None admin] PATH_INFO: `/v3/projects/e1b78198b2b24bfd8f607348b26438ad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:34.370139 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a887e7c6-4dd3-474d-8708-ebb0a9d47fa3 None admin] RBAC: Authorizing `identity:get_project(project_id=e1b78198b2b24bfd8f607348b26438ad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:34.371779 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a887e7c6-4dd3-474d-8708-ebb0a9d47fa3 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:34.373462 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 93/182] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 13:59:34 2022] GET /identity/v3/projects/e1b78198b2b24bfd8f607348b26438ad => generated 312 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:34.374225 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887e7c6-4dd3-474d-8708-ebb0a9d47fa3 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:34.423839 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:34.437238 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 90/183] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:34.438532 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a863e56-ef40-4f2f-957c-d443e1a9810e None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:34.524256 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887e7c6-4dd3-474d-8708-ebb0a9d47fa3 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:34.550354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a863e56-ef40-4f2f-957c-d443e1a9810e None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:34.559180 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:34.566397 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc05ea1e-aacb-4402-b46a-63535cf55a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:34.567787 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05ea1e-aacb-4402-b46a-63535cf55a84 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:34.568560 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05ea1e-aacb-4402-b46a-63535cf55a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:34.569049 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05ea1e-aacb-4402-b46a-63535cf55a84 None admin] PATH_INFO: `/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:34.570605 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc05ea1e-aacb-4402-b46a-63535cf55a84 None admin] RBAC: Authorizing `identity:get_project(project_id=e10c65c7a17c44a7a8711e7701efdcd0)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:34.572521 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc05ea1e-aacb-4402-b46a-63535cf55a84 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:34.575704 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 94/184] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 13:59:34 2022] GET /identity/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:34.577106 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc05ea1e-aacb-4402-b46a-63535cf55a84 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:34.589182 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c78b54-44cf-4812-86f2-267295b1e98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:34.589790 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c78b54-44cf-4812-86f2-267295b1e98c None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:34.590063 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c78b54-44cf-4812-86f2-267295b1e98c None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:34.590326 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c78b54-44cf-4812-86f2-267295b1e98c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:34.591077 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c78b54-44cf-4812-86f2-267295b1e98c None admin] RBAC: Adding query filter params (role.id=d0a6f7c5c34a4925b9efb7dd4ac915fc, scope.project.id=e1b78198b2b24bfd8f607348b26438ad, user.id=96b89bc850f04d96bedaea747c2c46b7) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:34.591380 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c78b54-44cf-4812-86f2-267295b1e98c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:34.592930 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c78b54-44cf-4812-86f2-267295b1e98c None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:34.603109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 91/185] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 13:59:34 2022] GET /identity/v3/role_assignments?user.id=96b89bc850f04d96bedaea747c2c46b7&scope.project.id=e1b78198b2b24bfd8f607348b26438ad&role.id=d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:34.604009 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c78b54-44cf-4812-86f2-267295b1e98c None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:34.704022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:34.725197 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc05ea1e-aacb-4402-b46a-63535cf55a84 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:34.753670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c78b54-44cf-4812-86f2-267295b1e98c None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:34.762362 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab4294-2963-420f-a98e-1d3cf3a86e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:34.763036 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab4294-2963-420f-a98e-1d3cf3a86e5d None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:34.763314 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab4294-2963-420f-a98e-1d3cf3a86e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:34.763579 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab4294-2963-420f-a98e-1d3cf3a86e5d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:34.774967 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fab4294-2963-420f-a98e-1d3cf3a86e5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:34.780473 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-8fab4294-2963-420f-a98e-1d3cf3a86e5d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 13:59:34.780473 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:34.780473 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:34.780473 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:34.780473 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:34.780473 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:34.780473 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:34.780473 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:34.780473 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:34.780473 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:34.780473 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:34.780473 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:34.780473 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:34.780473 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:34.780473 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:34.780473 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:34.781349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:34.781349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:34.781349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:34.781349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:34.781349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:34.781349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:34.781349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:34.781349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:34.781349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:34.781349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:34.781349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:34.781349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:34.781349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:34.781349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:34.781349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:34.781349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:34.782298 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:34.782298 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:34.782298 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:34.782298 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:34.782298 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:34.782298 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:34.782298 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:34.782298 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:34.782298 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:34.782298 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:34.782298 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 13:59:34.782298 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 13:59:34.782298 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 95/186] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 12 13:59:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 13:59:34.782990 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab4294-2963-420f-a98e-1d3cf3a86e5d None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:34.798263 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ca8ae5-a4e1-4cf5-b87f-20f655c71a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:34.799238 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ca8ae5-a4e1-4cf5-b87f-20f655c71a27 None admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:34.799719 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ca8ae5-a4e1-4cf5-b87f-20f655c71a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:34.799990 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ca8ae5-a4e1-4cf5-b87f-20f655c71a27 None admin] PATH_INFO: `/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0/users/c007b65736674f0c980027d5463e5710/roles/dab439aae69348a183b07971883cebfb` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:34.802041 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ca8ae5-a4e1-4cf5-b87f-20f655c71a27 None admin] RBAC: Authorizing `identity:create_grant(project_id=e10c65c7a17c44a7a8711e7701efdcd0, user_id=c007b65736674f0c980027d5463e5710, role_id=dab439aae69348a183b07971883cebfb)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:34.803527 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ca8ae5-a4e1-4cf5-b87f-20f655c71a27 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:34.820369 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 92/187] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 12 13:59:34 2022] PUT /identity/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0/users/c007b65736674f0c980027d5463e5710/roles/dab439aae69348a183b07971883cebfb => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 13:59:34.821201 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ca8ae5-a4e1-4cf5-b87f-20f655c71a27 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:34.866913 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:34.924140 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab4294-2963-420f-a98e-1d3cf3a86e5d None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:34.953608 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ca8ae5-a4e1-4cf5-b87f-20f655c71a27 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:34.964543 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e45f3ae-6e0f-4252-91ba-b52f1786e574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:34.965220 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45f3ae-6e0f-4252-91ba-b52f1786e574 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:34.965498 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45f3ae-6e0f-4252-91ba-b52f1786e574 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:34.965769 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45f3ae-6e0f-4252-91ba-b52f1786e574 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:34.973865 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:34.978555 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e45f3ae-6e0f-4252-91ba-b52f1786e574 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:34.983795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-7e45f3ae-6e0f-4252-91ba-b52f1786e574 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 13:59:34.983795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:34.983795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:34.983795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:34.983795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:34.983795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:34.983795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:34.983795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:34.983795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:34.983795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:34.983795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:34.983795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:34.983795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:34.983795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:34.983795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:34.983795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:34.984739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:34.984739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:34.984739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:34.984739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:34.984739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:34.984739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:34.984739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:34.984739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:34.984739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:34.984739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:34.984739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:34.984739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:34.984739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:34.984739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:34.984739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:34.984739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:34.989502 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:34.989502 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:34.989502 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:34.989502 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:34.989502 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:34.989502 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:34.989502 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:34.989502 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:34.989502 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:34.989502 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:34.989502 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 13:59:34.989502 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 13:59:34.989502 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 96/188] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 12 13:59:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 13:59:34.989502 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e45f3ae-6e0f-4252-91ba-b52f1786e574 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:35.014588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cbc31a-5801-4d93-a742-1ed9228161ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:35.015271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cbc31a-5801-4d93-a742-1ed9228161ea None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:35.015545 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cbc31a-5801-4d93-a742-1ed9228161ea None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:35.015807 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cbc31a-5801-4d93-a742-1ed9228161ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:35.016941 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cbc31a-5801-4d93-a742-1ed9228161ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:35.018370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cbc31a-5801-4d93-a742-1ed9228161ea None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:35.020175 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 93/189] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 13:59:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:35.020726 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cbc31a-5801-4d93-a742-1ed9228161ea None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:35.101798 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e45f3ae-6e0f-4252-91ba-b52f1786e574 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:35.119548 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:35.128357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cbc31a-5801-4d93-a742-1ed9228161ea None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:35.137327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90034cd-ec1e-42b7-af0c-ac310f69c5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:35.137942 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90034cd-ec1e-42b7-af0c-ac310f69c5b9 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:35.138219 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90034cd-ec1e-42b7-af0c-ac310f69c5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:35.138487 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90034cd-ec1e-42b7-af0c-ac310f69c5b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:35.139182 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90034cd-ec1e-42b7-af0c-ac310f69c5b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:35.139484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90034cd-ec1e-42b7-af0c-ac310f69c5b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:35.140839 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90034cd-ec1e-42b7-af0c-ac310f69c5b9 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:35.148732 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 97/190] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 13:59:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:35.149506 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90034cd-ec1e-42b7-af0c-ac310f69c5b9 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:35.170646 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2940eb94-1af3-48e0-a350-d2594361da4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:35.171287 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940eb94-1af3-48e0-a350-d2594361da4e None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:35.171564 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940eb94-1af3-48e0-a350-d2594361da4e None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:35.171830 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940eb94-1af3-48e0-a350-d2594361da4e None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:35.172517 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2940eb94-1af3-48e0-a350-d2594361da4e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:35.172814 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2940eb94-1af3-48e0-a350-d2594361da4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:35.174236 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2940eb94-1af3-48e0-a350-d2594361da4e None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:35.182568 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 94/191] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 13:59:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:35.183478 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2940eb94-1af3-48e0-a350-d2594361da4e None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:35.232544 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:35.263649 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90034cd-ec1e-42b7-af0c-ac310f69c5b9 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:35.291864 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2940eb94-1af3-48e0-a350-d2594361da4e None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:35.298492 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ea934e-b1e5-4eae-96cd-3d241a908143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:35.299133 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea934e-b1e5-4eae-96cd-3d241a908143 None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:35.299412 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea934e-b1e5-4eae-96cd-3d241a908143 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:35.299680 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea934e-b1e5-4eae-96cd-3d241a908143 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:35.300328 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ea934e-b1e5-4eae-96cd-3d241a908143 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:35.301845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ea934e-b1e5-4eae-96cd-3d241a908143 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:35.330518 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 98/192] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Sep 12 13:59:35 2022] POST /identity/v3/users => generated 297 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 13:59:35.331533 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ea934e-b1e5-4eae-96cd-3d241a908143 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:35.340028 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34c5b0-2f50-4fa4-a3c2-ce81033847ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:35.340028 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34c5b0-2f50-4fa4-a3c2-ce81033847ea None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:35.340028 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34c5b0-2f50-4fa4-a3c2-ce81033847ea None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:35.340028 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34c5b0-2f50-4fa4-a3c2-ce81033847ea None admin] PATH_INFO: `/v3/users/0240aefc9276455ea667ee27680ae777` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:35.360644 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee34c5b0-2f50-4fa4-a3c2-ce81033847ea None admin] RBAC: Authorizing `identity:get_user(user_id=0240aefc9276455ea667ee27680ae777)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:35.363728 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee34c5b0-2f50-4fa4-a3c2-ce81033847ea None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:35.366892 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 95/193] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 13:59:35 2022] GET /identity/v3/users/0240aefc9276455ea667ee27680ae777 => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:35.373023 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34c5b0-2f50-4fa4-a3c2-ce81033847ea None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:35.394381 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:35.486060 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ea934e-b1e5-4eae-96cd-3d241a908143 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:35.524389 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34c5b0-2f50-4fa4-a3c2-ce81033847ea None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:35.528327 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:35.530143 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a974245f-16ef-465c-8ab4-a0fbb39263e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:35.530808 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974245f-16ef-465c-8ab4-a0fbb39263e5 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:35.531162 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974245f-16ef-465c-8ab4-a0fbb39263e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:35.531496 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974245f-16ef-465c-8ab4-a0fbb39263e5 None admin] PATH_INFO: `/v3/users/0240aefc9276455ea667ee27680ae777` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:35.532613 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a974245f-16ef-465c-8ab4-a0fbb39263e5 None admin] RBAC: Authorizing `identity:get_user(user_id=0240aefc9276455ea667ee27680ae777)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:35.534059 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a974245f-16ef-465c-8ab4-a0fbb39263e5 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:35.535957 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 99/194] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 13:59:35 2022] GET /identity/v3/users/0240aefc9276455ea667ee27680ae777 => generated 283 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:35.541195 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a974245f-16ef-465c-8ab4-a0fbb39263e5 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:35.570499 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1cd57c-dcb2-4376-b9e0-7bf3966c3894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:35.571146 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1cd57c-dcb2-4376-b9e0-7bf3966c3894 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:35.571418 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1cd57c-dcb2-4376-b9e0-7bf3966c3894 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:35.571683 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1cd57c-dcb2-4376-b9e0-7bf3966c3894 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:35.654290 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1cd57c-dcb2-4376-b9e0-7bf3966c3894 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:35.655914 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1cd57c-dcb2-4376-b9e0-7bf3966c3894 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:35.676723 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a974245f-16ef-465c-8ab4-a0fbb39263e5 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:35.681728 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:35.725101 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a687c-3694-44ac-bccb-5fee6559d843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:35.725828 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a687c-3694-44ac-bccb-5fee6559d843 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:35.726114 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a687c-3694-44ac-bccb-5fee6559d843 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:35.726387 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a687c-3694-44ac-bccb-5fee6559d843 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:35.785909 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 96/195] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 13:59:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 13:59:35.791484 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1cd57c-dcb2-4376-b9e0-7bf3966c3894 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:35.795310 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:35.814316 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2a687c-3694-44ac-bccb-5fee6559d843 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:35.816027 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2a687c-3694-44ac-bccb-5fee6559d843 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:35.947050 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 100/196] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 13:59:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 12 13:59:35.950950 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1cd57c-dcb2-4376-b9e0-7bf3966c3894 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:35.955236 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a687c-3694-44ac-bccb-5fee6559d843 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:35.964477 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:36.033904 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e25ecc-b742-4bfd-844a-3150428cb642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:36.034594 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e25ecc-b742-4bfd-844a-3150428cb642 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:36.034896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e25ecc-b742-4bfd-844a-3150428cb642 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:36.035172 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e25ecc-b742-4bfd-844a-3150428cb642 None admin] PATH_INFO: `/v3/projects/6606f4e0816344eca4228a4e9d0ed52e` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:36.036429 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e25ecc-b742-4bfd-844a-3150428cb642 None admin] RBAC: Authorizing `identity:get_project(project_id=6606f4e0816344eca4228a4e9d0ed52e)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:36.038738 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e25ecc-b742-4bfd-844a-3150428cb642 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:36.043822 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 97/197] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 13:59:35 2022] GET /identity/v3/projects/6606f4e0816344eca4228a4e9d0ed52e => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 13:59:36.050130 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e25ecc-b742-4bfd-844a-3150428cb642 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:36.150284 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a687c-3694-44ac-bccb-5fee6559d843 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:36.167572 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:36.210392 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e25ecc-b742-4bfd-844a-3150428cb642 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:36.225184 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583efc0d-58c0-4dfb-b7ed-bed64ce9c1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:36.226156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583efc0d-58c0-4dfb-b7ed-bed64ce9c1e3 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:36.226631 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583efc0d-58c0-4dfb-b7ed-bed64ce9c1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:36.227123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583efc0d-58c0-4dfb-b7ed-bed64ce9c1e3 None admin] PATH_INFO: `/v3/projects/6606f4e0816344eca4228a4e9d0ed52e` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:36.230181 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583efc0d-58c0-4dfb-b7ed-bed64ce9c1e3 None admin] RBAC: Authorizing `identity:get_project(project_id=6606f4e0816344eca4228a4e9d0ed52e)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:36.231916 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583efc0d-58c0-4dfb-b7ed-bed64ce9c1e3 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:36.233850 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 101/198] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 13:59:35 2022] GET /identity/v3/projects/6606f4e0816344eca4228a4e9d0ed52e => generated 302 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 13:59:36.247450 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583efc0d-58c0-4dfb-b7ed-bed64ce9c1e3 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:36.286683 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa44afc-afb2-4858-bf84-1171aed28b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:36.287661 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa44afc-afb2-4858-bf84-1171aed28b12 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:36.288132 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa44afc-afb2-4858-bf84-1171aed28b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:36.288591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa44afc-afb2-4858-bf84-1171aed28b12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:36.289508 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa44afc-afb2-4858-bf84-1171aed28b12 None admin] RBAC: Adding query filter params (role.id=e14564f00daf48a28d4e32da8cd2efad, scope.project.id=6606f4e0816344eca4228a4e9d0ed52e, user.id=0240aefc9276455ea667ee27680ae777) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:36.290024 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa44afc-afb2-4858-bf84-1171aed28b12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:36.292870 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa44afc-afb2-4858-bf84-1171aed28b12 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:36.309509 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 98/199] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 13:59:36 2022] GET /identity/v3/role_assignments?user.id=0240aefc9276455ea667ee27680ae777&scope.project.id=6606f4e0816344eca4228a4e9d0ed52e&role.id=e14564f00daf48a28d4e32da8cd2efad => generated 259 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:36.346994 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:36.379935 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583efc0d-58c0-4dfb-b7ed-bed64ce9c1e3 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:36.417139 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfbfcb9-856d-498c-b1ed-92e08a1c2603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:36.418160 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbfcb9-856d-498c-b1ed-92e08a1c2603 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:36.418638 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbfcb9-856d-498c-b1ed-92e08a1c2603 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:36.419141 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbfcb9-856d-498c-b1ed-92e08a1c2603 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:36.420106 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfbfcb9-856d-498c-b1ed-92e08a1c2603 None admin] RBAC: Adding query filter params (role.id=bcde1b76af8048deb2829c5aaa27296c, scope.project.id=6606f4e0816344eca4228a4e9d0ed52e, user.id=0240aefc9276455ea667ee27680ae777) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:36.420618 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfbfcb9-856d-498c-b1ed-92e08a1c2603 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:36.422285 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfbfcb9-856d-498c-b1ed-92e08a1c2603 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:36.428899 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 102/200] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 13:59:36 2022] GET /identity/v3/role_assignments?user.id=0240aefc9276455ea667ee27680ae777&scope.project.id=6606f4e0816344eca4228a4e9d0ed52e&role.id=bcde1b76af8048deb2829c5aaa27296c => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:37.235767 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e30d7-dfee-444b-b1ac-0b2ccd69a37a None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:37.236260 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e30d7-dfee-444b-b1ac-0b2ccd69a37a None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:37.236260 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e30d7-dfee-444b-b1ac-0b2ccd69a37a None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:37.237872 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 99/201] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:37.242982 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c48338-33a1-4789-9c7a-3c9fd6601678 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:37.243362 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c48338-33a1-4789-9c7a-3c9fd6601678 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:37.243538 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c48338-33a1-4789-9c7a-3c9fd6601678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:37.267095 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-73c48338-33a1-4789-9c7a-3c9fd6601678 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:37.268665 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-73c48338-33a1-4789-9c7a-3c9fd6601678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:37.343737 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 103/202] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:37.794981 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4e30d7-dfee-444b-b1ac-0b2ccd69a37a None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:37.940447 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4e30d7-dfee-444b-b1ac-0b2ccd69a37a None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:37.988175 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f7534e-85da-4ff4-81b7-2f643e14e91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:37.988807 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7534e-85da-4ff4-81b7-2f643e14e91c None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:37.989080 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7534e-85da-4ff4-81b7-2f643e14e91c None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:37.989349 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7534e-85da-4ff4-81b7-2f643e14e91c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:37.992779 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:38.005724 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f7534e-85da-4ff4-81b7-2f643e14e91c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:38.011187 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-37f7534e-85da-4ff4-81b7-2f643e14e91c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 13:59:38.011187 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:38.011187 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:38.011187 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:38.011187 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:38.011187 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:38.011187 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:38.011187 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:38.011187 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:38.011187 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:38.011187 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:38.011187 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:38.011187 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:38.011187 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:38.011187 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:38.011187 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:38.012061 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:38.012061 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:38.012061 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:38.012061 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:38.012061 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:38.012061 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:38.012061 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:38.012061 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:38.012061 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:38.012061 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:38.012061 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:38.012061 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:38.012061 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:38.012061 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:38.012061 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:38.012061 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:38.012831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:38.012831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:38.012831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:38.012831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:38.012831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:38.012831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:38.012831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:38.012831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:38.012831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:38.012831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:38.012831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 13:59:38.012831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 13:59:38.012831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 100/203] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 13:59:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 13:59:38.021899 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c48338-33a1-4789-9c7a-3c9fd6601678 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:38.121474 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:38.165234 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c48338-33a1-4789-9c7a-3c9fd6601678 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:38.205950 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4616889-d4d5-4dd0-83f5-1a6a55006823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:38.206559 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4616889-d4d5-4dd0-83f5-1a6a55006823 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:38.206847 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4616889-d4d5-4dd0-83f5-1a6a55006823 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:38.207127 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4616889-d4d5-4dd0-83f5-1a6a55006823 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:38.207786 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4616889-d4d5-4dd0-83f5-1a6a55006823 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:38.208082 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4616889-d4d5-4dd0-83f5-1a6a55006823 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:38.209436 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4616889-d4d5-4dd0-83f5-1a6a55006823 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:38.217618 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 104/204] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 12 13:59:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:38.223246 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f7534e-85da-4ff4-81b7-2f643e14e91c None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:38.251528 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa5400-1037-4812-89dd-cd221a68a7ce None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:38.252017 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa5400-1037-4812-89dd-cd221a68a7ce None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:38.252121 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa5400-1037-4812-89dd-cd221a68a7ce None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:38.253626 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 105/205] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:38.260122 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2be0be-1562-43f8-a9c8-108b2a31000c None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:38.260444 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2be0be-1562-43f8-a9c8-108b2a31000c None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:38.260732 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2be0be-1562-43f8-a9c8-108b2a31000c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:38.288400 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-db2be0be-1562-43f8-a9c8-108b2a31000c None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:38.289580 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-db2be0be-1562-43f8-a9c8-108b2a31000c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:38.343119 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f7534e-85da-4ff4-81b7-2f643e14e91c None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:38.379158 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84cd20-cbd5-4a73-a94f-892514a82cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:38.379718 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84cd20-cbd5-4a73-a94f-892514a82cda None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:38.379992 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84cd20-cbd5-4a73-a94f-892514a82cda None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:38.380252 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84cd20-cbd5-4a73-a94f-892514a82cda None admin] PATH_INFO: `/v3/users/c007b65736674f0c980027d5463e5710` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:38.381590 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb84cd20-cbd5-4a73-a94f-892514a82cda None admin] RBAC: Authorizing `identity:get_user(user_id=c007b65736674f0c980027d5463e5710)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:38.383217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb84cd20-cbd5-4a73-a94f-892514a82cda None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:38.385382 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 101/206] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 13:59:38 2022] GET /identity/v3/users/c007b65736674f0c980027d5463e5710 => generated 289 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:38.390440 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84cd20-cbd5-4a73-a94f-892514a82cda None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:38.460433 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:38.469752 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 106/207] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:38.515727 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84cd20-cbd5-4a73-a94f-892514a82cda None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:38.536539 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b63f7-cb26-453c-94cd-c066e262e378 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:38.537152 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b63f7-cb26-453c-94cd-c066e262e378 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:38.537692 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b63f7-cb26-453c-94cd-c066e262e378 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:38.540593 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 107/208] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 13:59:38.545455 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578be24-2e06-4702-8cc9-14ced150c6ad None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:38.545969 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578be24-2e06-4702-8cc9-14ced150c6ad None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:38.546446 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578be24-2e06-4702-8cc9-14ced150c6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:38.568732 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-5578be24-2e06-4702-8cc9-14ced150c6ad None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:38.570576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-5578be24-2e06-4702-8cc9-14ced150c6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:38.572254 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6550b0-e3ae-4fe6-9dc4-1e2066927a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:38.573712 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6550b0-e3ae-4fe6-9dc4-1e2066927a6f None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:38.574206 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6550b0-e3ae-4fe6-9dc4-1e2066927a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:38.574675 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6550b0-e3ae-4fe6-9dc4-1e2066927a6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:38.668801 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:38.703024 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 108/209] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:38.704305 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6550b0-e3ae-4fe6-9dc4-1e2066927a6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:38.705944 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6550b0-e3ae-4fe6-9dc4-1e2066927a6f None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:38.790870 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:38.838266 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 102/210] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 13:59:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 13:59:38.844219 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5578be24-2e06-4702-8cc9-14ced150c6ad None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:38.894873 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6550b0-e3ae-4fe6-9dc4-1e2066927a6f None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:39.007760 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5578be24-2e06-4702-8cc9-14ced150c6ad None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:39.047976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d101c6c0-b0f8-4e20-b6b7-f399faa0da14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:39.048616 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101c6c0-b0f8-4e20-b6b7-f399faa0da14 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:39.048891 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101c6c0-b0f8-4e20-b6b7-f399faa0da14 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:39.049158 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101c6c0-b0f8-4e20-b6b7-f399faa0da14 None admin] PATH_INFO: `/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:39.050257 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d101c6c0-b0f8-4e20-b6b7-f399faa0da14 None admin] RBAC: Authorizing `identity:get_project(project_id=e10c65c7a17c44a7a8711e7701efdcd0)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:39.051747 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d101c6c0-b0f8-4e20-b6b7-f399faa0da14 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:39.053359 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 109/211] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 13:59:38 2022] GET /identity/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:39.054454 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1cb23-ddd1-4bf0-9cb2-f50b6db71a0d None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:39.054733 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1cb23-ddd1-4bf0-9cb2-f50b6db71a0d None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:39.055032 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1cb23-ddd1-4bf0-9cb2-f50b6db71a0d None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:39.056221 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 110/212] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:39.059012 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b57314-7ccd-40ff-bc8d-f0af113fe485 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:39.059012 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b57314-7ccd-40ff-bc8d-f0af113fe485 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:39.059012 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b57314-7ccd-40ff-bc8d-f0af113fe485 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:39.059648 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 111/213] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:39.062877 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b57314-7ccd-40ff-bc8d-f0af113fe485 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:39.063850 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6550b0-e3ae-4fe6-9dc4-1e2066927a6f None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:39.098035 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e85ba-5952-49c0-ae95-d1eb00a8d5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:39.098677 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e85ba-5952-49c0-ae95-d1eb00a8d5db None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:39.098992 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e85ba-5952-49c0-ae95-d1eb00a8d5db None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:39.099264 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e85ba-5952-49c0-ae95-d1eb00a8d5db None admin] PATH_INFO: `/v3/roles/member` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:39.109485 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e85ba-5952-49c0-ae95-d1eb00a8d5db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:39.114351 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-a73e85ba-5952-49c0-ae95-d1eb00a8d5db None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 13:59:39.114351 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:39.114351 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:39.114351 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:39.114351 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:39.114351 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:39.114351 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:39.114351 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:39.114351 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:39.114351 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:39.114351 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:39.114351 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:39.114351 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:39.114351 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:39.114351 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:39.114351 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:39.115341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:39.115341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:39.115341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:39.115341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:39.115341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:39.115341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:39.115341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:39.115341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:39.115341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:39.115341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:39.115341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:39.115341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:39.115341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:39.115341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:39.115341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:39.115341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:39.116142 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:39.116142 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:39.116142 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:39.116142 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:39.116142 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:39.116142 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:39.116142 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:39.116142 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:39.116142 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:39.116142 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:39.116142 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 13:59:39.116142 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 13:59:39.116142 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 103/214] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 13:59:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 13:59:39.116781 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e85ba-5952-49c0-ae95-d1eb00a8d5db None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:39.118295 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:39.141680 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:39.163418 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b57314-7ccd-40ff-bc8d-f0af113fe485 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:39.197034 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c0b97-9300-4be9-b250-ae05c16b81ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:39.198261 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c0b97-9300-4be9-b250-ae05c16b81ad None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:39.198540 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c0b97-9300-4be9-b250-ae05c16b81ad None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:39.198807 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c0b97-9300-4be9-b250-ae05c16b81ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:39.199935 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c0b97-9300-4be9-b250-ae05c16b81ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:39.201362 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c0b97-9300-4be9-b250-ae05c16b81ad None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:39.202946 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 112/215] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 13:59:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:39.204175 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad18d8-ee85-4771-a995-53260688669b None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:39.204458 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad18d8-ee85-4771-a995-53260688669b None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:39.204739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad18d8-ee85-4771-a995-53260688669b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:39.216399 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e85ba-5952-49c0-ae95-d1eb00a8d5db None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:39.222229 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-a9ad18d8-ee85-4771-a995-53260688669b None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:39.223448 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-a9ad18d8-ee85-4771-a995-53260688669b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:39.249083 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37bf9b-771b-4051-96a3-0aae114b6585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:39.249668 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37bf9b-771b-4051-96a3-0aae114b6585 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:39.249938 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37bf9b-771b-4051-96a3-0aae114b6585 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:39.250202 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37bf9b-771b-4051-96a3-0aae114b6585 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:39.250948 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf37bf9b-771b-4051-96a3-0aae114b6585 None admin] RBAC: Adding query filter params (role.id=dab439aae69348a183b07971883cebfb, scope.project.id=e10c65c7a17c44a7a8711e7701efdcd0, user.id=c007b65736674f0c980027d5463e5710) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:39.251245 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf37bf9b-771b-4051-96a3-0aae114b6585 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:39.252668 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf37bf9b-771b-4051-96a3-0aae114b6585 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:39.270943 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 104/216] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 13:59:39 2022] GET /identity/v3/role_assignments?user.id=c007b65736674f0c980027d5463e5710&scope.project.id=e10c65c7a17c44a7a8711e7701efdcd0&role.id=dab439aae69348a183b07971883cebfb => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:39.270943 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7966d54-d43e-43e1-ba5e-09a555900ea4 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:39.270943 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7966d54-d43e-43e1-ba5e-09a555900ea4 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:39.270943 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7966d54-d43e-43e1-ba5e-09a555900ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:39.281158 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-d7966d54-d43e-43e1-ba5e-09a555900ea4 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:39.282404 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-d7966d54-d43e-43e1-ba5e-09a555900ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:39.294878 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 113/217] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:39.296054 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad18d8-ee85-4771-a995-53260688669b None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:39.381207 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:39.383714 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 105/218] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:39.384835 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7966d54-d43e-43e1-ba5e-09a555900ea4 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:39.430478 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:39.448050 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad18d8-ee85-4771-a995-53260688669b None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:39.495962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd634a51-d645-40c7-adf4-0274a3be9fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:39.496986 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd634a51-d645-40c7-adf4-0274a3be9fee None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:39.497478 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd634a51-d645-40c7-adf4-0274a3be9fee None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:39.497991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd634a51-d645-40c7-adf4-0274a3be9fee None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:39.498890 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd634a51-d645-40c7-adf4-0274a3be9fee None admin] RBAC: Adding query filter params (name=member) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:39.499380 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd634a51-d645-40c7-adf4-0274a3be9fee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:39.500916 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd634a51-d645-40c7-adf4-0274a3be9fee None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:39.512424 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 114/219] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 12 13:59:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:39.518685 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd634a51-d645-40c7-adf4-0274a3be9fee None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:39.542531 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7966d54-d43e-43e1-ba5e-09a555900ea4 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:39.580351 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c5f03-088f-42af-a555-47526fb9ff04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:39.580959 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c5f03-088f-42af-a555-47526fb9ff04 None admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:39.581236 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c5f03-088f-42af-a555-47526fb9ff04 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:39.581611 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c5f03-088f-42af-a555-47526fb9ff04 None admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:39.582160 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c5f03-088f-42af-a555-47526fb9ff04 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:39.583902 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c5f03-088f-42af-a555-47526fb9ff04 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:39.612893 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 106/220] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Sep 12 13:59:39 2022] POST /identity/v3/users => generated 297 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 13:59:39.613884 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c5f03-088f-42af-a555-47526fb9ff04 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:39.637051 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd634a51-d645-40c7-adf4-0274a3be9fee None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:39.655962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:39.677335 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce698a-62b3-4958-95b5-ce8dd149d9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:39.678083 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce698a-62b3-4958-95b5-ce8dd149d9c8 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:39.678363 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce698a-62b3-4958-95b5-ce8dd149d9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:39.678634 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce698a-62b3-4958-95b5-ce8dd149d9c8 None admin] PATH_INFO: `/v3/users/4f7f313cdae247c9898ae49d2bfa6a35` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:39.703243 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ce698a-62b3-4958-95b5-ce8dd149d9c8 None admin] RBAC: Authorizing `identity:get_user(user_id=4f7f313cdae247c9898ae49d2bfa6a35)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:39.704790 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ce698a-62b3-4958-95b5-ce8dd149d9c8 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:39.706926 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 115/221] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 13:59:39 2022] GET /identity/v3/users/4f7f313cdae247c9898ae49d2bfa6a35 => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:39.707697 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce698a-62b3-4958-95b5-ce8dd149d9c8 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:39.714722 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:39.743140 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c5f03-088f-42af-a555-47526fb9ff04 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:39.782598 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4acb6c-b528-43b1-9917-21e60dc84221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:39.783278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4acb6c-b528-43b1-9917-21e60dc84221 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:39.783558 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4acb6c-b528-43b1-9917-21e60dc84221 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:39.783828 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4acb6c-b528-43b1-9917-21e60dc84221 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:39.797787 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4acb6c-b528-43b1-9917-21e60dc84221 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:39.803217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-4c4acb6c-b528-43b1-9917-21e60dc84221 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 13:59:39.803217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:39.803217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:39.803217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:39.803217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:39.803217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:39.803217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:39.803217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:39.803217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:39.803217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:39.803217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:39.803217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:39.803217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:39.803217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:39.803217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:39.803217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:39.804102 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:39.804102 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:39.804102 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:39.804102 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:39.804102 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:39.804102 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:39.804102 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:39.804102 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:39.804102 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:39.804102 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:39.804102 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:39.804102 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:39.804102 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:39.804102 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:39.804102 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:39.804102 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:39.808335 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:39.808335 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:39.808335 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:39.808335 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:39.808335 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:39.808335 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:39.808335 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:39.808335 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:39.808335 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:39.808335 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:39.808335 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 13:59:39.808335 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 13:59:39.808335 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 107/222] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 12 13:59:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 13:59:39.808335 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4acb6c-b528-43b1-9917-21e60dc84221 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:39.836133 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce698a-62b3-4958-95b5-ce8dd149d9c8 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:39.874186 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9e6f2-163a-4559-acd2-7b0cff5e45b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:39.874824 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9e6f2-163a-4559-acd2-7b0cff5e45b8 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:39.875152 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9e6f2-163a-4559-acd2-7b0cff5e45b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:39.875422 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9e6f2-163a-4559-acd2-7b0cff5e45b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:39.889848 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a9e6f2-163a-4559-acd2-7b0cff5e45b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:39.897227 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-33a9e6f2-163a-4559-acd2-7b0cff5e45b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 13:59:39.897227 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:39.897227 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:39.897227 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:39.897227 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:39.897227 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:39.897227 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:39.897227 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:39.897227 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:39.897227 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:39.897227 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:39.897227 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:39.897227 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:39.897227 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:39.897227 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:39.897227 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:39.898137 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:39.898137 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:39.898137 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:39.898137 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:39.898137 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:39.898137 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:39.898137 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:39.898137 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:39.898137 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:39.898137 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:39.898137 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:39.898137 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:39.898137 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:39.898137 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:39.898137 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:39.898137 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:39.899324 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:39.899324 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:39.899324 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:39.899324 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:39.899324 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:39.899324 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:39.899324 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:39.899324 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:39.899324 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:39.899324 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:39.899324 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 13:59:39.899324 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 13:59:39.899324 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 116/223] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 12 13:59:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 13:59:39.899324 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9e6f2-163a-4559-acd2-7b0cff5e45b8 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:39.935929 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4acb6c-b528-43b1-9917-21e60dc84221 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:39.939870 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:39.976997 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e7f0e9-e68a-469b-bb67-6793a1591fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:39.977611 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7f0e9-e68a-469b-bb67-6793a1591fd7 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:39.977888 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7f0e9-e68a-469b-bb67-6793a1591fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:39.978157 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7f0e9-e68a-469b-bb67-6793a1591fd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:39.981534 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:40.028935 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9e6f2-163a-4559-acd2-7b0cff5e45b8 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:40.054345 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e7f0e9-e68a-469b-bb67-6793a1591fd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:40.055748 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e7f0e9-e68a-469b-bb67-6793a1591fd7 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:40.070493 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab33ac6-1f56-4b31-82c9-ac7a1350d144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:40.071486 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab33ac6-1f56-4b31-82c9-ac7a1350d144 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:40.072025 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab33ac6-1f56-4b31-82c9-ac7a1350d144 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:40.072292 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab33ac6-1f56-4b31-82c9-ac7a1350d144 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:40.073619 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab33ac6-1f56-4b31-82c9-ac7a1350d144 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:40.073924 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab33ac6-1f56-4b31-82c9-ac7a1350d144 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:40.075477 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab33ac6-1f56-4b31-82c9-ac7a1350d144 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:40.086260 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 117/224] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 13:59:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:40.091384 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab33ac6-1f56-4b31-82c9-ac7a1350d144 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:40.185752 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 108/225] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 13:59:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 13:59:40.186862 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e7f0e9-e68a-469b-bb67-6793a1591fd7 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:40.218286 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:40.225406 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab33ac6-1f56-4b31-82c9-ac7a1350d144 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:40.249990 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:40.268354 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7cdb44-181a-40c9-803c-e2d9c274744c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:40.268967 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7cdb44-181a-40c9-803c-e2d9c274744c None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:40.269246 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7cdb44-181a-40c9-803c-e2d9c274744c None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:40.269519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7cdb44-181a-40c9-803c-e2d9c274744c None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:40.270187 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7cdb44-181a-40c9-803c-e2d9c274744c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:40.270484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7cdb44-181a-40c9-803c-e2d9c274744c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:40.271909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7cdb44-181a-40c9-803c-e2d9c274744c None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:40.286922 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 118/226] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 13:59:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:40.287614 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7cdb44-181a-40c9-803c-e2d9c274744c None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:40.333320 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e7f0e9-e68a-469b-bb67-6793a1591fd7 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:40.369602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a58c5-eb2e-40b1-81fc-7ea37a5744f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:40.370608 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a58c5-eb2e-40b1-81fc-7ea37a5744f1 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:40.371122 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a58c5-eb2e-40b1-81fc-7ea37a5744f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:40.371599 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a58c5-eb2e-40b1-81fc-7ea37a5744f1 None admin] PATH_INFO: `/v3/users/0240aefc9276455ea667ee27680ae777` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:40.373163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a58c5-eb2e-40b1-81fc-7ea37a5744f1 None admin] RBAC: Authorizing `identity:get_user(user_id=0240aefc9276455ea667ee27680ae777)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:40.374813 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a58c5-eb2e-40b1-81fc-7ea37a5744f1 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:40.376687 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 109/227] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 13:59:40 2022] GET /identity/v3/users/0240aefc9276455ea667ee27680ae777 => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:40.377676 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a58c5-eb2e-40b1-81fc-7ea37a5744f1 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:40.409085 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7cdb44-181a-40c9-803c-e2d9c274744c None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:40.448685 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89194855-5082-4f92-b233-7228f5ac04b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:40.449327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89194855-5082-4f92-b233-7228f5ac04b5 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:40.449603 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89194855-5082-4f92-b233-7228f5ac04b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:40.449875 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89194855-5082-4f92-b233-7228f5ac04b5 None admin] PATH_INFO: `/v3/projects/6606f4e0816344eca4228a4e9d0ed52e` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:40.451020 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89194855-5082-4f92-b233-7228f5ac04b5 None admin] RBAC: Authorizing `identity:get_project(project_id=6606f4e0816344eca4228a4e9d0ed52e)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:40.453098 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89194855-5082-4f92-b233-7228f5ac04b5 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:40.455054 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 119/228] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 13:59:40 2022] GET /identity/v3/projects/6606f4e0816344eca4228a4e9d0ed52e => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:40.459677 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89194855-5082-4f92-b233-7228f5ac04b5 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:40.490503 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a58c5-eb2e-40b1-81fc-7ea37a5744f1 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:40.492301 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:40.493006 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:40.529751 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b29f9-6c30-4efc-b81e-f31e485b41a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:40.530786 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b29f9-6c30-4efc-b81e-f31e485b41a9 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:40.531305 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b29f9-6c30-4efc-b81e-f31e485b41a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:40.531766 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b29f9-6c30-4efc-b81e-f31e485b41a9 None admin] PATH_INFO: `/v3/users/0240aefc9276455ea667ee27680ae777` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:40.533084 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13b29f9-6c30-4efc-b81e-f31e485b41a9 None admin] RBAC: Authorizing `identity:get_user(user_id=0240aefc9276455ea667ee27680ae777)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:40.534717 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13b29f9-6c30-4efc-b81e-f31e485b41a9 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:40.536517 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 110/229] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 13:59:40 2022] GET /identity/v3/users/0240aefc9276455ea667ee27680ae777 => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:40.541928 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b29f9-6c30-4efc-b81e-f31e485b41a9 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:40.575319 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89194855-5082-4f92-b233-7228f5ac04b5 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:40.612410 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb862d98-669b-4080-89a3-ebb03d58e6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:40.612961 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb862d98-669b-4080-89a3-ebb03d58e6ef None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:40.613233 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb862d98-669b-4080-89a3-ebb03d58e6ef None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:40.613502 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb862d98-669b-4080-89a3-ebb03d58e6ef None admin] PATH_INFO: `/v3/projects/6606f4e0816344eca4228a4e9d0ed52e` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:40.614659 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb862d98-669b-4080-89a3-ebb03d58e6ef None admin] RBAC: Authorizing `identity:get_project(project_id=6606f4e0816344eca4228a4e9d0ed52e)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:40.616067 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb862d98-669b-4080-89a3-ebb03d58e6ef None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:40.617976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 120/230] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 13:59:40 2022] GET /identity/v3/projects/6606f4e0816344eca4228a4e9d0ed52e => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:40.618578 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb862d98-669b-4080-89a3-ebb03d58e6ef None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:40.655789 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b29f9-6c30-4efc-b81e-f31e485b41a9 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:40.693703 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c09331-fc04-4197-9f20-dfcef1cb8efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:40.694648 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c09331-fc04-4197-9f20-dfcef1cb8efc None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:40.695168 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c09331-fc04-4197-9f20-dfcef1cb8efc None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:40.695650 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c09331-fc04-4197-9f20-dfcef1cb8efc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:40.696614 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c09331-fc04-4197-9f20-dfcef1cb8efc None admin] RBAC: Adding query filter params (role.id=d0a6f7c5c34a4925b9efb7dd4ac915fc, scope.project.id=6606f4e0816344eca4228a4e9d0ed52e, user.id=4f7f313cdae247c9898ae49d2bfa6a35) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:40.697398 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c09331-fc04-4197-9f20-dfcef1cb8efc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:40.699620 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c09331-fc04-4197-9f20-dfcef1cb8efc None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:40.705495 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 111/231] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 13:59:40 2022] GET /identity/v3/role_assignments?user.id=4f7f313cdae247c9898ae49d2bfa6a35&scope.project.id=6606f4e0816344eca4228a4e9d0ed52e&role.id=d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:40.706519 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c09331-fc04-4197-9f20-dfcef1cb8efc None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:40.766900 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:40.781654 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb862d98-669b-4080-89a3-ebb03d58e6ef None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:40.794402 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:40.840570 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd3326-7899-4e12-b92a-86fa3c6bb511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:40.841646 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd3326-7899-4e12-b92a-86fa3c6bb511 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:40.842724 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd3326-7899-4e12-b92a-86fa3c6bb511 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:40.843225 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd3326-7899-4e12-b92a-86fa3c6bb511 None admin] PATH_INFO: `/v3/projects/6606f4e0816344eca4228a4e9d0ed52e` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:40.844628 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd3326-7899-4e12-b92a-86fa3c6bb511 None admin] RBAC: Authorizing `identity:get_project(project_id=6606f4e0816344eca4228a4e9d0ed52e)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:40.846377 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd3326-7899-4e12-b92a-86fa3c6bb511 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:40.848690 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 121/232] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 13:59:40 2022] GET /identity/v3/projects/6606f4e0816344eca4228a4e9d0ed52e => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:40.855961 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd3326-7899-4e12-b92a-86fa3c6bb511 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:40.942564 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c09331-fc04-4197-9f20-dfcef1cb8efc None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:40.989908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e361a-f503-4cbb-a932-55fd5572b016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:40.990974 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e361a-f503-4cbb-a932-55fd5572b016 None admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:40.991473 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e361a-f503-4cbb-a932-55fd5572b016 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:40.991938 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e361a-f503-4cbb-a932-55fd5572b016 None admin] PATH_INFO: `/v3/projects/6606f4e0816344eca4228a4e9d0ed52e/users/0240aefc9276455ea667ee27680ae777/roles/bcde1b76af8048deb2829c5aaa27296c` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:41.000797 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2e361a-f503-4cbb-a932-55fd5572b016 None admin] RBAC: Authorizing `identity:create_grant(project_id=6606f4e0816344eca4228a4e9d0ed52e, user_id=0240aefc9276455ea667ee27680ae777, role_id=bcde1b76af8048deb2829c5aaa27296c)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:41.002685 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2e361a-f503-4cbb-a932-55fd5572b016 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:41.004674 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd3326-7899-4e12-b92a-86fa3c6bb511 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:41.018645 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 112/233] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 12 13:59:40 2022] PUT /identity/v3/projects/6606f4e0816344eca4228a4e9d0ed52e/users/0240aefc9276455ea667ee27680ae777/roles/bcde1b76af8048deb2829c5aaa27296c => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 13:59:41.046034 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0393a9-d744-45db-8bbd-d08ee1b54e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:41.047000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0393a9-d744-45db-8bbd-d08ee1b54e84 None admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:41.047426 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0393a9-d744-45db-8bbd-d08ee1b54e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:41.047874 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0393a9-d744-45db-8bbd-d08ee1b54e84 None admin] PATH_INFO: `/v3/projects/6606f4e0816344eca4228a4e9d0ed52e/users/0240aefc9276455ea667ee27680ae777/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:41.050140 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0393a9-d744-45db-8bbd-d08ee1b54e84 None admin] RBAC: Authorizing `identity:create_grant(project_id=6606f4e0816344eca4228a4e9d0ed52e, user_id=0240aefc9276455ea667ee27680ae777, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:41.052019 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0393a9-d744-45db-8bbd-d08ee1b54e84 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:41.079463 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 122/234] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 12 13:59:40 2022] PUT /identity/v3/projects/6606f4e0816344eca4228a4e9d0ed52e/users/0240aefc9276455ea667ee27680ae777/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 13:59:42.206070 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9c544-e313-4160-a015-b017be433a53 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:42.206572 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9c544-e313-4160-a015-b017be433a53 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:42.206572 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9c544-e313-4160-a015-b017be433a53 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:42.208185 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 113/235] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:42.213590 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de1274-6538-4d00-8513-80f4603d8af4 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:42.214017 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de1274-6538-4d00-8513-80f4603d8af4 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:42.214114 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de1274-6538-4d00-8513-80f4603d8af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:42.240054 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-83de1274-6538-4d00-8513-80f4603d8af4 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:42.241302 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-83de1274-6538-4d00-8513-80f4603d8af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:42.267429 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:42.330232 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 123/236] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:42.353317 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1acba-1683-4db7-ae82-586fad36ecaf None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:42.353633 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1acba-1683-4db7-ae82-586fad36ecaf None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:42.353913 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1acba-1683-4db7-ae82-586fad36ecaf None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:42.355436 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 114/237] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:42.360315 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea95b9-ba04-475f-89c9-1eecd584f613 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:42.360606 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea95b9-ba04-475f-89c9-1eecd584f613 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:42.360879 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea95b9-ba04-475f-89c9-1eecd584f613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:42.388007 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-f2ea95b9-ba04-475f-89c9-1eecd584f613 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:42.389213 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f2ea95b9-ba04-475f-89c9-1eecd584f613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:42.478249 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 124/238] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:42.694674 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c1acba-1683-4db7-ae82-586fad36ecaf None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:42.798809 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:42.811385 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c1acba-1683-4db7-ae82-586fad36ecaf None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:42.826858 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ea95b9-ba04-475f-89c9-1eecd584f613 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:42.867734 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c246d-4227-42a2-ad45-3c057d4db9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:42.868904 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c246d-4227-42a2-ad45-3c057d4db9e4 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:42.870479 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c246d-4227-42a2-ad45-3c057d4db9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:42.871566 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c246d-4227-42a2-ad45-3c057d4db9e4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:42.893564 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9c246d-4227-42a2-ad45-3c057d4db9e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:42.901821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-8d9c246d-4227-42a2-ad45-3c057d4db9e4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 13:59:42.901821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:42.901821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:42.901821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:42.901821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:42.901821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:42.901821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:42.901821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:42.901821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:42.901821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:42.901821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:42.901821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:42.901821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:42.901821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:42.901821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:42.901821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:42.902681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:42.902681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:42.902681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:42.902681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:42.902681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:42.902681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:42.902681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:42.902681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:42.902681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:42.902681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:42.902681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:42.902681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:42.902681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:42.902681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:42.902681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:42.902681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:42.903511 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:42.903511 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:42.903511 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:42.903511 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:42.903511 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:42.903511 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:42.903511 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:42.903511 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:42.903511 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:42.903511 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:42.903511 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 13:59:42.903511 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 13:59:42.908316 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 115/239] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 13:59:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 13:59:42.912880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c246d-4227-42a2-ad45-3c057d4db9e4 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:42.974268 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:43.012788 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ea95b9-ba04-475f-89c9-1eecd584f613 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:43.075218 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d6ff3-376d-434f-a6ef-513f31df3207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:43.076217 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d6ff3-376d-434f-a6ef-513f31df3207 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:43.076746 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d6ff3-376d-434f-a6ef-513f31df3207 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:43.077228 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d6ff3-376d-434f-a6ef-513f31df3207 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:43.078698 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d6ff3-376d-434f-a6ef-513f31df3207 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:43.080536 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d6ff3-376d-434f-a6ef-513f31df3207 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:43.082672 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 125/240] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 13:59:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:43.087724 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d6ff3-376d-434f-a6ef-513f31df3207 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:43.121947 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c246d-4227-42a2-ad45-3c057d4db9e4 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:43.207556 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4bce2-5be5-42ee-842a-3c56719b78f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:43.208231 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4bce2-5be5-42ee-842a-3c56719b78f2 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:43.208496 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4bce2-5be5-42ee-842a-3c56719b78f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:43.208703 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4bce2-5be5-42ee-842a-3c56719b78f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:43.209377 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4bce2-5be5-42ee-842a-3c56719b78f2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:43.209675 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4bce2-5be5-42ee-842a-3c56719b78f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:43.213840 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4bce2-5be5-42ee-842a-3c56719b78f2 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:43.230009 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 116/241] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 12 13:59:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:43.242660 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4bce2-5be5-42ee-842a-3c56719b78f2 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:43.261879 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d6ff3-376d-434f-a6ef-513f31df3207 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:43.272166 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:43.311957 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef502d9-32ff-4e1f-b109-18af27b1f31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:43.313442 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef502d9-32ff-4e1f-b109-18af27b1f31b None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:43.314004 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef502d9-32ff-4e1f-b109-18af27b1f31b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:43.314596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef502d9-32ff-4e1f-b109-18af27b1f31b None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:43.315795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef502d9-32ff-4e1f-b109-18af27b1f31b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:43.317832 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef502d9-32ff-4e1f-b109-18af27b1f31b None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:43.345941 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 126/242] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Sep 12 13:59:43 2022] POST /identity/v3/users => generated 293 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 13:59:43.347760 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9f6d2-ed38-4b45-9d0f-00c7e32a67a5 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:43.348238 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9f6d2-ed38-4b45-9d0f-00c7e32a67a5 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:43.348708 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9f6d2-ed38-4b45-9d0f-00c7e32a67a5 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:43.350580 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 127/243] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:43.361943 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142de34-1555-4a5a-834e-b63fd5e04074 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:43.362613 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142de34-1555-4a5a-834e-b63fd5e04074 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:43.363148 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142de34-1555-4a5a-834e-b63fd5e04074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:43.371871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4bce2-5be5-42ee-842a-3c56719b78f2 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:43.384544 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-6142de34-1555-4a5a-834e-b63fd5e04074 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:43.386183 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-6142de34-1555-4a5a-834e-b63fd5e04074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:43.398874 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:43.433392 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d78ecf6-875e-4f75-b856-349270c08618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:43.434510 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78ecf6-875e-4f75-b856-349270c08618 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:43.435056 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78ecf6-875e-4f75-b856-349270c08618 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:43.435620 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78ecf6-875e-4f75-b856-349270c08618 None admin] PATH_INFO: `/v3/users/3b71b8fbb18747eb87004d1157273501` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:43.467438 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d78ecf6-875e-4f75-b856-349270c08618 None admin] RBAC: Authorizing `identity:get_user(user_id=3b71b8fbb18747eb87004d1157273501)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:43.471189 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d78ecf6-875e-4f75-b856-349270c08618 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:43.476132 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 117/244] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 13:59:43 2022] GET /identity/v3/users/3b71b8fbb18747eb87004d1157273501 => generated 297 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 13:59:43.482585 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d78ecf6-875e-4f75-b856-349270c08618 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:43.548099 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 128/245] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 13:59:43.611201 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:43.638523 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d78ecf6-875e-4f75-b856-349270c08618 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:43.675762 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8aa5bc-0430-42c2-b313-d6440fab76c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:43.676720 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8aa5bc-0430-42c2-b313-d6440fab76c3 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:43.677236 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8aa5bc-0430-42c2-b313-d6440fab76c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:43.677748 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8aa5bc-0430-42c2-b313-d6440fab76c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:43.761898 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8aa5bc-0430-42c2-b313-d6440fab76c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:43.763411 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8aa5bc-0430-42c2-b313-d6440fab76c3 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:43.813362 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c72b5d-56e9-4642-a4c8-793c874d5708 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:43.813673 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c72b5d-56e9-4642-a4c8-793c874d5708 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:43.813933 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c72b5d-56e9-4642-a4c8-793c874d5708 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:43.815691 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 129/246] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 13:59:43.825737 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f3f90-3641-4bcd-a4f3-a9f4d3678da2 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:43.826020 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f3f90-3641-4bcd-a4f3-a9f4d3678da2 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:43.826292 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f3f90-3641-4bcd-a4f3-a9f4d3678da2 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:43.836307 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 130/247] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:43 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:43.837514 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b149872-3d9f-4f23-9753-0969e4ed7955 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:43.837803 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b149872-3d9f-4f23-9753-0969e4ed7955 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:43.838616 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b149872-3d9f-4f23-9753-0969e4ed7955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:43.857090 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:43.862671 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-2b149872-3d9f-4f23-9753-0969e4ed7955 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:43.862671 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-2b149872-3d9f-4f23-9753-0969e4ed7955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:43.885838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 118/248] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 13:59:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 13:59:43.887714 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8c226-4362-406e-9dca-f5e00bd01668 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:43.887994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8c226-4362-406e-9dca-f5e00bd01668 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:43.888263 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8c226-4362-406e-9dca-f5e00bd01668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:43.906810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-3cc8c226-4362-406e-9dca-f5e00bd01668 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:43.908071 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-3cc8c226-4362-406e-9dca-f5e00bd01668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:43.955969 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 131/249] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:43.956667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b149872-3d9f-4f23-9753-0969e4ed7955 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:44.002920 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 119/250] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:44.004124 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8c226-4362-406e-9dca-f5e00bd01668 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:44.058547 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:44.121295 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b149872-3d9f-4f23-9753-0969e4ed7955 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:44.177218 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8c226-4362-406e-9dca-f5e00bd01668 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:44.180015 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:44.188804 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1099e3f-a03f-46de-9ec6-e44bcee1229b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:44.189447 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1099e3f-a03f-46de-9ec6-e44bcee1229b None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:44.189739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1099e3f-a03f-46de-9ec6-e44bcee1229b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:44.189992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1099e3f-a03f-46de-9ec6-e44bcee1229b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:44.207784 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1099e3f-a03f-46de-9ec6-e44bcee1229b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:44.212670 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-b1099e3f-a03f-46de-9ec6-e44bcee1229b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 13:59:44.212670 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:44.212670 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:44.212670 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:44.212670 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:44.212670 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:44.212670 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:44.212670 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:44.212670 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:44.212670 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:44.212670 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:44.212670 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:44.212670 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:44.212670 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:44.212670 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:44.212670 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:44.213576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:44.213576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:44.213576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:44.213576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:44.213576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:44.213576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:44.213576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:44.213576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:44.213576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:44.213576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:44.213576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:44.213576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:44.213576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:44.213576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:44.213576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:44.213576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:44.214376 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:44.214376 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:44.214376 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:44.214376 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:44.214376 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:44.214376 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:44.214376 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:44.214376 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:44.214376 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:44.214376 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:44.214376 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 13:59:44.214376 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 13:59:44.214376 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 132/251] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 13:59:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 13:59:44.223808 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1099e3f-a03f-46de-9ec6-e44bcee1229b None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:44.236694 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2131bc0-238a-4237-98a0-c0c04f7c10c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:44.237963 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2131bc0-238a-4237-98a0-c0c04f7c10c5 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:44.238429 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2131bc0-238a-4237-98a0-c0c04f7c10c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:44.238906 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2131bc0-238a-4237-98a0-c0c04f7c10c5 None admin] PATH_INFO: `/v3/projects/6606f4e0816344eca4228a4e9d0ed52e` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:44.241214 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2131bc0-238a-4237-98a0-c0c04f7c10c5 None admin] RBAC: Authorizing `identity:get_project(project_id=6606f4e0816344eca4228a4e9d0ed52e)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:44.242928 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2131bc0-238a-4237-98a0-c0c04f7c10c5 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:44.245805 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 120/252] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 13:59:44 2022] GET /identity/v3/projects/6606f4e0816344eca4228a4e9d0ed52e => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:44.251221 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2131bc0-238a-4237-98a0-c0c04f7c10c5 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:44.369104 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2131bc0-238a-4237-98a0-c0c04f7c10c5 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:44.407016 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b44bc26-4cd3-4756-a1b6-f9a5fb24067c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:44.407551 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44bc26-4cd3-4756-a1b6-f9a5fb24067c None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:44.407823 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44bc26-4cd3-4756-a1b6-f9a5fb24067c None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:44.408082 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44bc26-4cd3-4756-a1b6-f9a5fb24067c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:44.409010 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b44bc26-4cd3-4756-a1b6-f9a5fb24067c None admin] RBAC: Adding query filter params (role.id=dab439aae69348a183b07971883cebfb, scope.project.id=6606f4e0816344eca4228a4e9d0ed52e, user.id=3b71b8fbb18747eb87004d1157273501) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:44.409194 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b44bc26-4cd3-4756-a1b6-f9a5fb24067c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:44.410612 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b44bc26-4cd3-4756-a1b6-f9a5fb24067c None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:44.417485 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 121/253] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 13:59:44 2022] GET /identity/v3/role_assignments?user.id=3b71b8fbb18747eb87004d1157273501&scope.project.id=6606f4e0816344eca4228a4e9d0ed52e&role.id=dab439aae69348a183b07971883cebfb => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 13:59:44.418377 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b44bc26-4cd3-4756-a1b6-f9a5fb24067c None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:44.427148 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1099e3f-a03f-46de-9ec6-e44bcee1229b None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:44.457100 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:44.458900 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:44.484695 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad67699-76da-4053-9548-843e178cb39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:44.485294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad67699-76da-4053-9548-843e178cb39a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:44.485671 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad67699-76da-4053-9548-843e178cb39a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:44.486033 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad67699-76da-4053-9548-843e178cb39a None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:44.486796 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad67699-76da-4053-9548-843e178cb39a None admin] RBAC: Adding query filter params (name=member) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:44.487212 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad67699-76da-4053-9548-843e178cb39a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:44.488694 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad67699-76da-4053-9548-843e178cb39a None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:44.503837 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 133/254] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 12 13:59:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:44.509113 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad67699-76da-4053-9548-843e178cb39a None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:44.539117 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b44bc26-4cd3-4756-a1b6-f9a5fb24067c None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:44.575509 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474c700-c6bf-4435-ab54-7436d8ecfc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:44.576521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474c700-c6bf-4435-ab54-7436d8ecfc05 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:44.576990 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474c700-c6bf-4435-ab54-7436d8ecfc05 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:44.577454 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474c700-c6bf-4435-ab54-7436d8ecfc05 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:44.589542 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8474c700-c6bf-4435-ab54-7436d8ecfc05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:44.595590 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-8474c700-c6bf-4435-ab54-7436d8ecfc05 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 13:59:44.595590 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:44.595590 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:44.595590 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:44.595590 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:44.595590 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:44.595590 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:44.595590 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:44.595590 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:44.595590 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:44.595590 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:44.595590 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:44.595590 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:44.595590 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:44.595590 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:44.595590 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:44.596464 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:44.596464 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:44.596464 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:44.596464 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:44.596464 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:44.596464 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:44.596464 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:44.596464 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:44.596464 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:44.596464 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:44.596464 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:44.596464 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:44.596464 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:44.596464 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:44.596464 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:44.596464 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:44.597249 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:44.597249 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:44.597249 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:44.597249 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:44.597249 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:44.597249 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:44.597249 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:44.597249 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:44.597249 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:44.597249 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:44.597249 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 13:59:44.597249 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 13:59:44.599806 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 122/255] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 12 13:59:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 13:59:44.604965 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474c700-c6bf-4435-ab54-7436d8ecfc05 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:44.698877 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad67699-76da-4053-9548-843e178cb39a None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:44.707254 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:44.712758 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474c700-c6bf-4435-ab54-7436d8ecfc05 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:44.750756 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8793d6c-02b6-4ed4-9431-ed82bd02fd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:44.751805 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8793d6c-02b6-4ed4-9431-ed82bd02fd56 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:44.752286 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8793d6c-02b6-4ed4-9431-ed82bd02fd56 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:44.752778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8793d6c-02b6-4ed4-9431-ed82bd02fd56 None admin] PATH_INFO: `/v3/users/4f7f313cdae247c9898ae49d2bfa6a35` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:44.754313 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8793d6c-02b6-4ed4-9431-ed82bd02fd56 None admin] RBAC: Authorizing `identity:get_user(user_id=4f7f313cdae247c9898ae49d2bfa6a35)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:44.756090 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8793d6c-02b6-4ed4-9431-ed82bd02fd56 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:44.758127 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 123/256] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 13:59:44 2022] GET /identity/v3/users/4f7f313cdae247c9898ae49d2bfa6a35 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:44.759228 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8793d6c-02b6-4ed4-9431-ed82bd02fd56 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:44.781889 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455acaac-87a1-4c05-855a-3c1d103ce317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:44.783020 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455acaac-87a1-4c05-855a-3c1d103ce317 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:44.783496 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455acaac-87a1-4c05-855a-3c1d103ce317 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:44.783973 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455acaac-87a1-4c05-855a-3c1d103ce317 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:44.800928 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455acaac-87a1-4c05-855a-3c1d103ce317 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:44.807668 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-455acaac-87a1-4c05-855a-3c1d103ce317 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 13:59:44.807668 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:44.807668 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:44.807668 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:44.807668 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:44.807668 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:44.807668 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:44.807668 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:44.807668 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:44.807668 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:44.807668 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:44.807668 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:44.807668 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:44.807668 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:44.807668 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:44.807668 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:44.808561 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:44.808561 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:44.808561 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:44.808561 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:44.808561 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:44.808561 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:44.808561 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:44.808561 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:44.808561 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:44.808561 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:44.808561 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:44.808561 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:44.808561 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:44.808561 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:44.808561 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:44.808561 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:44.809358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:44.809358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:44.809358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:44.809358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:44.809358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:44.809358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:44.809358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:44.809358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:44.809358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:44.809358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:44.809358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 13:59:44.809358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 13:59:44.809358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 134/257] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 12 13:59:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 13:59:44.810005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455acaac-87a1-4c05-855a-3c1d103ce317 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:44.887495 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8793d6c-02b6-4ed4-9431-ed82bd02fd56 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:44.938197 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:44.946522 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4abed-376b-48ba-a881-1d49bd8f12f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:44.947993 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4abed-376b-48ba-a881-1d49bd8f12f3 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:44.948277 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4abed-376b-48ba-a881-1d49bd8f12f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:44.948557 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4abed-376b-48ba-a881-1d49bd8f12f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:44.950370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a4abed-376b-48ba-a881-1d49bd8f12f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:44.950617 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a4abed-376b-48ba-a881-1d49bd8f12f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:44.952016 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a4abed-376b-48ba-a881-1d49bd8f12f3 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:44.967300 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 124/258] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 13:59:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:44.968060 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4abed-376b-48ba-a881-1d49bd8f12f3 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:45.027164 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:45.040296 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455acaac-87a1-4c05-855a-3c1d103ce317 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:45.112874 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4abed-376b-48ba-a881-1d49bd8f12f3 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:45.147355 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227801ad-e743-4a06-94ca-06c64491ef3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:45.147971 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227801ad-e743-4a06-94ca-06c64491ef3f None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:45.148248 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227801ad-e743-4a06-94ca-06c64491ef3f None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:45.148511 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227801ad-e743-4a06-94ca-06c64491ef3f None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:45.149175 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227801ad-e743-4a06-94ca-06c64491ef3f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:45.149475 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227801ad-e743-4a06-94ca-06c64491ef3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:45.150999 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227801ad-e743-4a06-94ca-06c64491ef3f None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:45.159354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 125/259] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 13:59:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:45.160167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227801ad-e743-4a06-94ca-06c64491ef3f None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:45.163537 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef067c-2780-481b-a7cf-e0efa146c7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:45.164199 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef067c-2780-481b-a7cf-e0efa146c7fb None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:45.164479 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef067c-2780-481b-a7cf-e0efa146c7fb None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:45.164753 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef067c-2780-481b-a7cf-e0efa146c7fb None admin] PATH_INFO: `/v3/projects/6606f4e0816344eca4228a4e9d0ed52e` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:45.171560 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ef067c-2780-481b-a7cf-e0efa146c7fb None admin] RBAC: Authorizing `identity:get_project(project_id=6606f4e0816344eca4228a4e9d0ed52e)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:45.173032 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ef067c-2780-481b-a7cf-e0efa146c7fb None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:45.174990 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 135/260] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 13:59:44 2022] GET /identity/v3/projects/6606f4e0816344eca4228a4e9d0ed52e => generated 302 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:45.175774 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef067c-2780-481b-a7cf-e0efa146c7fb None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:45.270816 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:45.295797 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227801ad-e743-4a06-94ca-06c64491ef3f None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:45.296062 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef067c-2780-481b-a7cf-e0efa146c7fb None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:45.325421 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:45.365634 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd75ef6-c3ff-47cb-a3cc-d8f761ed9eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:45.366272 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd75ef6-c3ff-47cb-a3cc-d8f761ed9eea None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:45.366550 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd75ef6-c3ff-47cb-a3cc-d8f761ed9eea None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:45.366822 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd75ef6-c3ff-47cb-a3cc-d8f761ed9eea None admin] PATH_INFO: `/v3/users/0240aefc9276455ea667ee27680ae777` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:45.368395 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd75ef6-c3ff-47cb-a3cc-d8f761ed9eea None admin] RBAC: Authorizing `identity:get_user(user_id=0240aefc9276455ea667ee27680ae777)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:45.370648 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8854be6c-ba1d-4f5a-a5d6-b890fcc39b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:45.371309 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8854be6c-ba1d-4f5a-a5d6-b890fcc39b24 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:45.371584 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8854be6c-ba1d-4f5a-a5d6-b890fcc39b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:45.371855 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8854be6c-ba1d-4f5a-a5d6-b890fcc39b24 None admin] PATH_INFO: `/v3/users/0240aefc9276455ea667ee27680ae777` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:45.372839 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd75ef6-c3ff-47cb-a3cc-d8f761ed9eea None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:45.374637 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 136/261] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 13:59:45 2022] GET /identity/v3/users/0240aefc9276455ea667ee27680ae777 => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:45.375205 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd75ef6-c3ff-47cb-a3cc-d8f761ed9eea None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:45.376106 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8854be6c-ba1d-4f5a-a5d6-b890fcc39b24 None admin] RBAC: Authorizing `identity:get_user(user_id=0240aefc9276455ea667ee27680ae777)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:45.377538 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8854be6c-ba1d-4f5a-a5d6-b890fcc39b24 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:45.379370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 126/262] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 13:59:45 2022] GET /identity/v3/users/0240aefc9276455ea667ee27680ae777 => generated 283 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:45.387722 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8854be6c-ba1d-4f5a-a5d6-b890fcc39b24 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:45.487729 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd75ef6-c3ff-47cb-a3cc-d8f761ed9eea None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:45.523848 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54595083-08f7-45c5-b6c6-6943dbd1d6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:45.524842 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54595083-08f7-45c5-b6c6-6943dbd1d6dd None admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:45.525328 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54595083-08f7-45c5-b6c6-6943dbd1d6dd None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:45.525849 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54595083-08f7-45c5-b6c6-6943dbd1d6dd None admin] PATH_INFO: `/v3/projects/6606f4e0816344eca4228a4e9d0ed52e/users/4f7f313cdae247c9898ae49d2bfa6a35/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:45.528019 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54595083-08f7-45c5-b6c6-6943dbd1d6dd None admin] RBAC: Authorizing `identity:create_grant(project_id=6606f4e0816344eca4228a4e9d0ed52e, user_id=4f7f313cdae247c9898ae49d2bfa6a35, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:45.529699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54595083-08f7-45c5-b6c6-6943dbd1d6dd None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:45.563008 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 137/263] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 12 13:59:45 2022] PUT /identity/v3/projects/6606f4e0816344eca4228a4e9d0ed52e/users/4f7f313cdae247c9898ae49d2bfa6a35/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 13:59:45.564349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54595083-08f7-45c5-b6c6-6943dbd1d6dd None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:45.572161 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8854be6c-ba1d-4f5a-a5d6-b890fcc39b24 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:45.617290 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd6185-a991-4e95-8649-7fabc68fd9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:45.617919 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd6185-a991-4e95-8649-7fabc68fd9f5 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:45.618194 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd6185-a991-4e95-8649-7fabc68fd9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:45.618461 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd6185-a991-4e95-8649-7fabc68fd9f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:45.656660 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:45.687270 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:45.748592 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54595083-08f7-45c5-b6c6-6943dbd1d6dd None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:45.765271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfd6185-a991-4e95-8649-7fabc68fd9f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:45.766749 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfd6185-a991-4e95-8649-7fabc68fd9f5 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:45.792239 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c11b3a8-3aa6-4e77-91c8-69627fe96284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:45.792239 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11b3a8-3aa6-4e77-91c8-69627fe96284 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:45.792593 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11b3a8-3aa6-4e77-91c8-69627fe96284 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:45.792893 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11b3a8-3aa6-4e77-91c8-69627fe96284 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:45.873825 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c11b3a8-3aa6-4e77-91c8-69627fe96284 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:45.875447 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c11b3a8-3aa6-4e77-91c8-69627fe96284 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:45.896083 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 127/264] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 13:59:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 13:59:45.904193 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd6185-a991-4e95-8649-7fabc68fd9f5 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:45.927632 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:45.981304 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 138/265] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 13:59:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 13:59:45.986223 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:45.986988 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c11b3a8-3aa6-4e77-91c8-69627fe96284 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:46.045924 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd6185-a991-4e95-8649-7fabc68fd9f5 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:46.095646 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c11b3a8-3aa6-4e77-91c8-69627fe96284 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:46.110367 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e778414-7200-459a-90a6-8e610e5deb52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:46.111346 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e778414-7200-459a-90a6-8e610e5deb52 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:46.111816 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e778414-7200-459a-90a6-8e610e5deb52 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:46.112366 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e778414-7200-459a-90a6-8e610e5deb52 None admin] PATH_INFO: `/v3/projects/6606f4e0816344eca4228a4e9d0ed52e` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:46.116509 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e778414-7200-459a-90a6-8e610e5deb52 None admin] RBAC: Authorizing `identity:get_project(project_id=6606f4e0816344eca4228a4e9d0ed52e)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:46.118147 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e778414-7200-459a-90a6-8e610e5deb52 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:46.121035 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 128/266] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 13:59:45 2022] GET /identity/v3/projects/6606f4e0816344eca4228a4e9d0ed52e => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:46.126488 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e778414-7200-459a-90a6-8e610e5deb52 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:46.130377 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d28ff76-ec50-4604-b9f3-fe16c7088cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:46.131093 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d28ff76-ec50-4604-b9f3-fe16c7088cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:46.131370 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d28ff76-ec50-4604-b9f3-fe16c7088cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:46.131639 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d28ff76-ec50-4604-b9f3-fe16c7088cc3 None admin] PATH_INFO: `/v3/projects/6606f4e0816344eca4228a4e9d0ed52e` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:46.132765 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d28ff76-ec50-4604-b9f3-fe16c7088cc3 None admin] RBAC: Authorizing `identity:get_project(project_id=6606f4e0816344eca4228a4e9d0ed52e)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:46.134182 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d28ff76-ec50-4604-b9f3-fe16c7088cc3 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:46.136571 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 139/267] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 13:59:45 2022] GET /identity/v3/projects/6606f4e0816344eca4228a4e9d0ed52e => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 13:59:46.142229 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d28ff76-ec50-4604-b9f3-fe16c7088cc3 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:46.159293 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:46.245199 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d28ff76-ec50-4604-b9f3-fe16c7088cc3 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:46.279020 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e778414-7200-459a-90a6-8e610e5deb52 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:46.308113 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:46.308959 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120b34a-4545-4e3e-a4cb-6c3018270276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:46.309590 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120b34a-4545-4e3e-a4cb-6c3018270276 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:46.309866 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120b34a-4545-4e3e-a4cb-6c3018270276 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:46.310129 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120b34a-4545-4e3e-a4cb-6c3018270276 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:46.310930 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5120b34a-4545-4e3e-a4cb-6c3018270276 None admin] RBAC: Adding query filter params (role.id=e14564f00daf48a28d4e32da8cd2efad, scope.project.id=6606f4e0816344eca4228a4e9d0ed52e, user.id=0240aefc9276455ea667ee27680ae777) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:46.311186 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5120b34a-4545-4e3e-a4cb-6c3018270276 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:46.312649 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5120b34a-4545-4e3e-a4cb-6c3018270276 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:46.318635 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2427d77-41b3-4bf5-8488-9456aee44da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:46.319532 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2427d77-41b3-4bf5-8488-9456aee44da4 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:46.320087 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2427d77-41b3-4bf5-8488-9456aee44da4 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:46.320546 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2427d77-41b3-4bf5-8488-9456aee44da4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:46.321467 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2427d77-41b3-4bf5-8488-9456aee44da4 None admin] RBAC: Adding query filter params (role.id=bcde1b76af8048deb2829c5aaa27296c, scope.project.id=6606f4e0816344eca4228a4e9d0ed52e, user.id=0240aefc9276455ea667ee27680ae777) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:46.322284 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2427d77-41b3-4bf5-8488-9456aee44da4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:46.324472 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2427d77-41b3-4bf5-8488-9456aee44da4 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:46.327238 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 140/268] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 13:59:46 2022] GET /identity/v3/role_assignments?user.id=0240aefc9276455ea667ee27680ae777&scope.project.id=6606f4e0816344eca4228a4e9d0ed52e&role.id=e14564f00daf48a28d4e32da8cd2efad => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 13:59:46.331522 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 129/269] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 13:59:46 2022] GET /identity/v3/role_assignments?user.id=0240aefc9276455ea667ee27680ae777&scope.project.id=6606f4e0816344eca4228a4e9d0ed52e&role.id=bcde1b76af8048deb2829c5aaa27296c => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 13:59:46.587714 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f8592-6148-4505-a92a-271c6512807e None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:46.589089 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f8592-6148-4505-a92a-271c6512807e None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:46.589361 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f8592-6148-4505-a92a-271c6512807e None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:46.590861 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 141/270] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:46.596651 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c31ba-d2e9-402f-847e-32e536237eb6 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:46.596942 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c31ba-d2e9-402f-847e-32e536237eb6 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:46.597196 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c31ba-d2e9-402f-847e-32e536237eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:46.620016 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-c89c31ba-d2e9-402f-847e-32e536237eb6 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:46.621268 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-c89c31ba-d2e9-402f-847e-32e536237eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:46.708504 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 130/271] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:46.748915 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b3696-ecbf-43dd-a379-bc6a792c04ca None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:46.749434 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b3696-ecbf-43dd-a379-bc6a792c04ca None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:46.749706 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b3696-ecbf-43dd-a379-bc6a792c04ca None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:46.751337 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 142/272] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:46.756704 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f4f541-7c45-47c1-a9e8-182b2ed66f73 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:46.756994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f4f541-7c45-47c1-a9e8-182b2ed66f73 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:46.757263 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f4f541-7c45-47c1-a9e8-182b2ed66f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:46.778401 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-b5f4f541-7c45-47c1-a9e8-182b2ed66f73 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:46.779610 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-b5f4f541-7c45-47c1-a9e8-182b2ed66f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:46.861563 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 131/273] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:47.006673 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b3696-ecbf-43dd-a379-bc6a792c04ca None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:47.091840 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:47.116350 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b3696-ecbf-43dd-a379-bc6a792c04ca None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:47.150949 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf99fe51-8c9f-4a81-87d7-88e85f0faaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:47.151585 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99fe51-8c9f-4a81-87d7-88e85f0faaa6 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:47.151863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99fe51-8c9f-4a81-87d7-88e85f0faaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:47.152133 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99fe51-8c9f-4a81-87d7-88e85f0faaa6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:47.162456 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf99fe51-8c9f-4a81-87d7-88e85f0faaa6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:47.168535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-cf99fe51-8c9f-4a81-87d7-88e85f0faaa6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 13:59:47.168535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:47.168535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:47.168535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:47.168535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:47.168535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:47.168535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:47.168535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:47.168535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:47.168535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:47.168535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:47.168535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:47.168535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:47.168535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:47.168535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:47.168535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:47.169373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:47.169373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:47.169373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:47.169373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:47.169373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:47.169373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:47.169373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:47.169373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:47.169373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:47.169373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:47.169373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:47.169373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:47.169373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:47.169373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:47.169373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:47.169373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:47.170149 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:47.170149 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:47.170149 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:47.170149 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:47.170149 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:47.170149 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:47.170149 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:47.170149 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:47.170149 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:47.170149 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:47.170149 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 13:59:47.170149 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 13:59:47.170149 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 143/274] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 13:59:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 13:59:47.175860 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f4f541-7c45-47c1-a9e8-182b2ed66f73 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:47.190780 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf99fe51-8c9f-4a81-87d7-88e85f0faaa6 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:47.243386 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:47.305953 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f4f541-7c45-47c1-a9e8-182b2ed66f73 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:47.307304 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf99fe51-8c9f-4a81-87d7-88e85f0faaa6 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:47.342523 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd6641-498c-4633-8c8d-342c46e6b4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:47.343525 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd6641-498c-4633-8c8d-342c46e6b4c0 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:47.344131 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd6641-498c-4633-8c8d-342c46e6b4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:47.344601 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd6641-498c-4633-8c8d-342c46e6b4c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:47.345469 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd6641-498c-4633-8c8d-342c46e6b4c0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:47.346073 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd6641-498c-4633-8c8d-342c46e6b4c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:47.347814 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd6641-498c-4633-8c8d-342c46e6b4c0 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:47.349190 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc6d99-0134-44fb-adc6-d2aa5819137d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:47.349987 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc6d99-0134-44fb-adc6-d2aa5819137d None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:47.350265 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc6d99-0134-44fb-adc6-d2aa5819137d None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:47.350535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc6d99-0134-44fb-adc6-d2aa5819137d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:47.351983 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc6d99-0134-44fb-adc6-d2aa5819137d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:47.353399 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc6d99-0134-44fb-adc6-d2aa5819137d None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:47.360640 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 144/275] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 13:59:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 13:59:47.361140 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 132/276] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 12 13:59:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:47.361958 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc6d99-0134-44fb-adc6-d2aa5819137d None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:47.369766 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:47.369909 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd6641-498c-4633-8c8d-342c46e6b4c0 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:47.482824 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc6d99-0134-44fb-adc6-d2aa5819137d None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:47.517503 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb88215f-9ca3-4e3b-8885-95f7d6b09b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:47.518143 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88215f-9ca3-4e3b-8885-95f7d6b09b16 None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:47.518418 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88215f-9ca3-4e3b-8885-95f7d6b09b16 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:47.518685 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88215f-9ca3-4e3b-8885-95f7d6b09b16 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:47.519132 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd6641-498c-4633-8c8d-342c46e6b4c0 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:47.519359 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb88215f-9ca3-4e3b-8885-95f7d6b09b16 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:47.520798 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb88215f-9ca3-4e3b-8885-95f7d6b09b16 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:47.531028 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:47.545720 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 145/277] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Sep 12 13:59:47 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 13:59:47.547096 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df3b8a-a947-4932-a935-72ea854a110c None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:47.547276 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df3b8a-a947-4932-a935-72ea854a110c None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:47.547507 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df3b8a-a947-4932-a935-72ea854a110c None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:47.549229 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 146/278] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:47.561704 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d90c0-5aec-40df-a940-4bc1c4c18b7f None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:47.562128 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d90c0-5aec-40df-a940-4bc1c4c18b7f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:47.562495 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d90c0-5aec-40df-a940-4bc1c4c18b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:47.565933 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ec9ca-641c-47c8-a1b4-a2a7e68644ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:47.566794 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ec9ca-641c-47c8-a1b4-a2a7e68644ad None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:47.567276 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ec9ca-641c-47c8-a1b4-a2a7e68644ad None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:47.567767 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ec9ca-641c-47c8-a1b4-a2a7e68644ad None admin] PATH_INFO: `/v3/users/0f4fc0e080a54b35b44b86553cec9320` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:47.583930 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-ff1d90c0-5aec-40df-a940-4bc1c4c18b7f None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:47.584272 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629ec9ca-641c-47c8-a1b4-a2a7e68644ad None admin] RBAC: Authorizing `identity:get_user(user_id=0f4fc0e080a54b35b44b86553cec9320)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:47.585400 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ff1d90c0-5aec-40df-a940-4bc1c4c18b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:47.586148 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629ec9ca-641c-47c8-a1b4-a2a7e68644ad None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:47.588809 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 133/279] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 13:59:47 2022] GET /identity/v3/users/0f4fc0e080a54b35b44b86553cec9320 => generated 293 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 13:59:47.595959 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ec9ca-641c-47c8-a1b4-a2a7e68644ad None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:47.667888 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:47.685458 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 147/280] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:47.720627 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ec9ca-641c-47c8-a1b4-a2a7e68644ad None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:47.758595 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ccaf0-c227-4527-a2f0-5ffb9d6e6b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:47.759242 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ccaf0-c227-4527-a2f0-5ffb9d6e6b75 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:47.759519 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ccaf0-c227-4527-a2f0-5ffb9d6e6b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:47.760329 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ccaf0-c227-4527-a2f0-5ffb9d6e6b75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:47.761056 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ccaf0-c227-4527-a2f0-5ffb9d6e6b75 None admin] RBAC: Adding query filter params (role.id=d0a6f7c5c34a4925b9efb7dd4ac915fc, user.id=0f4fc0e080a54b35b44b86553cec9320, scope.system=all) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:47.761361 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ccaf0-c227-4527-a2f0-5ffb9d6e6b75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:47.762825 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ccaf0-c227-4527-a2f0-5ffb9d6e6b75 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:47.767841 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 134/281] 10.222.0.42 () {56 vars in 1308 bytes} [Mon Sep 12 13:59:47 2022] GET /identity/v3/role_assignments?user.id=0f4fc0e080a54b35b44b86553cec9320&scope.system=all&role.id=d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:47.971971 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d90c0-5aec-40df-a940-4bc1c4c18b7f None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:48.101072 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d90c0-5aec-40df-a940-4bc1c4c18b7f None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:48.148931 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885cb260-6dea-48a8-b42d-7e85b2da4d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:48.149925 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885cb260-6dea-48a8-b42d-7e85b2da4d40 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:48.151317 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885cb260-6dea-48a8-b42d-7e85b2da4d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:48.151785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885cb260-6dea-48a8-b42d-7e85b2da4d40 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:48.169655 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885cb260-6dea-48a8-b42d-7e85b2da4d40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:48.174923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-885cb260-6dea-48a8-b42d-7e85b2da4d40 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 13:59:48.174923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:48.174923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:48.174923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:48.174923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:48.174923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:48.174923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:48.174923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:48.174923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:48.174923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:48.174923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:48.174923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:48.174923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:48.174923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:48.174923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:48.174923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:48.175785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:48.175785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:48.175785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:48.175785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:48.175785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:48.175785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:48.175785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:48.175785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:48.175785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:48.175785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:48.175785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:48.175785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:48.175785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:48.175785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:48.175785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:48.175785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:48.176557 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:48.176557 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:48.176557 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:48.176557 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:48.176557 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:48.176557 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:48.176557 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:48.176557 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:48.176557 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:48.176557 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:48.176557 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 13:59:48.176557 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 13:59:48.179783 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 148/282] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 13:59:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 13:59:48.184647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ccaf0-c227-4527-a2f0-5ffb9d6e6b75 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:48.249112 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:48.343851 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ccaf0-c227-4527-a2f0-5ffb9d6e6b75 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:48.392476 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272682b5-1ce0-4311-bd61-035c6a7e691e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:48.393092 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272682b5-1ce0-4311-bd61-035c6a7e691e None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:48.393366 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272682b5-1ce0-4311-bd61-035c6a7e691e None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:48.393634 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272682b5-1ce0-4311-bd61-035c6a7e691e None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:48.394306 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272682b5-1ce0-4311-bd61-035c6a7e691e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:48.394605 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272682b5-1ce0-4311-bd61-035c6a7e691e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:48.396055 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272682b5-1ce0-4311-bd61-035c6a7e691e None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:48.405692 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 135/283] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 12 13:59:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:48.411254 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885cb260-6dea-48a8-b42d-7e85b2da4d40 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:48.454322 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a16778-72e6-4039-be1f-48aa026385c8 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:48.454964 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a16778-72e6-4039-be1f-48aa026385c8 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:48.455431 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a16778-72e6-4039-be1f-48aa026385c8 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:48.458245 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 136/284] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 13:59:48.462445 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e041ad-4d27-4c6a-9ba4-7f576422b05b None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:48.463656 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e041ad-4d27-4c6a-9ba4-7f576422b05b None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:48.464202 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e041ad-4d27-4c6a-9ba4-7f576422b05b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:48.483545 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-34e041ad-4d27-4c6a-9ba4-7f576422b05b None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:48.485054 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-34e041ad-4d27-4c6a-9ba4-7f576422b05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:48.531214 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:48.582529 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 137/285] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 13:59:48.606849 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885cb260-6dea-48a8-b42d-7e85b2da4d40 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:48.643458 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9c00c-8da9-4139-826b-61002efd9d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:48.644090 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9c00c-8da9-4139-826b-61002efd9d33 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:48.644367 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9c00c-8da9-4139-826b-61002efd9d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:48.644638 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9c00c-8da9-4139-826b-61002efd9d33 None admin] PATH_INFO: `/v3/users/3b71b8fbb18747eb87004d1157273501` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:48.645775 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa9c00c-8da9-4139-826b-61002efd9d33 None admin] RBAC: Authorizing `identity:get_user(user_id=3b71b8fbb18747eb87004d1157273501)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:48.647213 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa9c00c-8da9-4139-826b-61002efd9d33 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:48.649217 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 149/286] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 13:59:48 2022] GET /identity/v3/users/3b71b8fbb18747eb87004d1157273501 => generated 297 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:48.654908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e041ad-4d27-4c6a-9ba4-7f576422b05b None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:48.689784 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:48.785470 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e041ad-4d27-4c6a-9ba4-7f576422b05b None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:48.835031 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae702b-b8f3-443f-9ca0-35aff41558b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:48.835753 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae702b-b8f3-443f-9ca0-35aff41558b9 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:48.836029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae702b-b8f3-443f-9ca0-35aff41558b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:48.836296 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae702b-b8f3-443f-9ca0-35aff41558b9 None admin] PATH_INFO: `/v3/projects/6606f4e0816344eca4228a4e9d0ed52e` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:48.837410 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ae702b-b8f3-443f-9ca0-35aff41558b9 None admin] RBAC: Authorizing `identity:get_project(project_id=6606f4e0816344eca4228a4e9d0ed52e)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:48.839460 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ae702b-b8f3-443f-9ca0-35aff41558b9 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:48.841420 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 138/287] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 13:59:48 2022] GET /identity/v3/projects/6606f4e0816344eca4228a4e9d0ed52e => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:48.850163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9c00c-8da9-4139-826b-61002efd9d33 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:48.953646 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae702b-b8f3-443f-9ca0-35aff41558b9 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:48.965662 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9c00c-8da9-4139-826b-61002efd9d33 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:48.984605 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:49.003299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141112bf-cff4-4247-bbe7-6d52fbafa08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:49.003948 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141112bf-cff4-4247-bbe7-6d52fbafa08c None admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:49.004234 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141112bf-cff4-4247-bbe7-6d52fbafa08c None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:49.004488 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141112bf-cff4-4247-bbe7-6d52fbafa08c None admin] PATH_INFO: `/v3/projects/6606f4e0816344eca4228a4e9d0ed52e/users/3b71b8fbb18747eb87004d1157273501/roles/dab439aae69348a183b07971883cebfb` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:49.006430 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141112bf-cff4-4247-bbe7-6d52fbafa08c None admin] RBAC: Authorizing `identity:create_grant(project_id=6606f4e0816344eca4228a4e9d0ed52e, user_id=3b71b8fbb18747eb87004d1157273501, role_id=dab439aae69348a183b07971883cebfb)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:49.007989 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141112bf-cff4-4247-bbe7-6d52fbafa08c None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:49.027554 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 150/288] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 12 13:59:48 2022] PUT /identity/v3/projects/6606f4e0816344eca4228a4e9d0ed52e/users/3b71b8fbb18747eb87004d1157273501/roles/dab439aae69348a183b07971883cebfb => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 13:59:49.100189 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:49.112844 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae702b-b8f3-443f-9ca0-35aff41558b9 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:49.157037 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a4914-03ef-4eda-8dd6-216fc967e21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:49.157672 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a4914-03ef-4eda-8dd6-216fc967e21f None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:49.157945 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a4914-03ef-4eda-8dd6-216fc967e21f None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:49.158212 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a4914-03ef-4eda-8dd6-216fc967e21f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:49.172999 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9a4914-03ef-4eda-8dd6-216fc967e21f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:49.178178 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-3a9a4914-03ef-4eda-8dd6-216fc967e21f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 13:59:49.178178 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:49.178178 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:49.178178 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:49.178178 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:49.178178 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:49.178178 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:49.178178 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:49.178178 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:49.178178 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:49.178178 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:49.178178 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:49.178178 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:49.178178 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:49.178178 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:49.178178 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:49.179073 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:49.179073 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:49.179073 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:49.179073 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:49.179073 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:49.179073 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:49.179073 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:49.179073 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:49.179073 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:49.179073 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:49.179073 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:49.179073 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:49.179073 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:49.179073 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:49.179073 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:49.179073 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:49.179886 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:49.179886 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:49.179886 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:49.179886 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:49.179886 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:49.179886 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:49.179886 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:49.179886 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:49.179886 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:49.179886 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:49.179886 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 13:59:49.179886 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 13:59:49.179886 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 139/289] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 13:59:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 13:59:49.185399 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141112bf-cff4-4247-bbe7-6d52fbafa08c None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:49.305445 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141112bf-cff4-4247-bbe7-6d52fbafa08c None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:49.345262 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a51b6-9f2b-43ca-8599-12886a49d0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:49.345887 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a51b6-9f2b-43ca-8599-12886a49d0a4 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:49.346162 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a51b6-9f2b-43ca-8599-12886a49d0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:49.346429 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a51b6-9f2b-43ca-8599-12886a49d0a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:49.347153 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9a51b6-9f2b-43ca-8599-12886a49d0a4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:49.347452 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9a51b6-9f2b-43ca-8599-12886a49d0a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:49.348888 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9a51b6-9f2b-43ca-8599-12886a49d0a4 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:49.359043 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 151/290] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 12 13:59:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:49.368216 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a4914-03ef-4eda-8dd6-216fc967e21f None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:49.485114 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a4914-03ef-4eda-8dd6-216fc967e21f None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:49.520479 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e51ba9-3ed8-4d93-a299-a5f8069ed462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:49.521107 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e51ba9-3ed8-4d93-a299-a5f8069ed462 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:49.521380 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e51ba9-3ed8-4d93-a299-a5f8069ed462 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:49.521642 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e51ba9-3ed8-4d93-a299-a5f8069ed462 None admin] PATH_INFO: `/v3/users/4f7f313cdae247c9898ae49d2bfa6a35` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:49.522747 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e51ba9-3ed8-4d93-a299-a5f8069ed462 None admin] RBAC: Authorizing `identity:get_user(user_id=4f7f313cdae247c9898ae49d2bfa6a35)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:49.524201 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e51ba9-3ed8-4d93-a299-a5f8069ed462 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:49.528086 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 140/291] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 13:59:49 2022] GET /identity/v3/users/4f7f313cdae247c9898ae49d2bfa6a35 => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:49.535954 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a51b6-9f2b-43ca-8599-12886a49d0a4 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:49.600844 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:49.671976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a51b6-9f2b-43ca-8599-12886a49d0a4 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:49.706421 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bcb68b-9a04-4c17-97fb-0e1130b12691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:49.707041 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bcb68b-9a04-4c17-97fb-0e1130b12691 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:49.707322 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bcb68b-9a04-4c17-97fb-0e1130b12691 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:49.707587 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bcb68b-9a04-4c17-97fb-0e1130b12691 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:49.778591 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bcb68b-9a04-4c17-97fb-0e1130b12691 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:49.780065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bcb68b-9a04-4c17-97fb-0e1130b12691 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:49.828325 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:49.897640 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 152/292] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 13:59:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 13:59:49.904118 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e51ba9-3ed8-4d93-a299-a5f8069ed462 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:49.938275 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:50.055355 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e51ba9-3ed8-4d93-a299-a5f8069ed462 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:50.102305 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeff6f8-d723-4fdb-9464-6296aa2689bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:50.102952 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeff6f8-d723-4fdb-9464-6296aa2689bd None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:50.103228 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeff6f8-d723-4fdb-9464-6296aa2689bd None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:50.104024 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeff6f8-d723-4fdb-9464-6296aa2689bd None admin] PATH_INFO: `/v3/projects/6606f4e0816344eca4228a4e9d0ed52e` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:50.107491 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeff6f8-d723-4fdb-9464-6296aa2689bd None admin] RBAC: Authorizing `identity:get_project(project_id=6606f4e0816344eca4228a4e9d0ed52e)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:50.108937 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeff6f8-d723-4fdb-9464-6296aa2689bd None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:50.110880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 141/293] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 13:59:49 2022] GET /identity/v3/projects/6606f4e0816344eca4228a4e9d0ed52e => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:50.116413 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bcb68b-9a04-4c17-97fb-0e1130b12691 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:50.260309 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bcb68b-9a04-4c17-97fb-0e1130b12691 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:50.313060 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519a7f3a-2886-4ece-90d3-16a5bb169416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:50.313698 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a7f3a-2886-4ece-90d3-16a5bb169416 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:50.313975 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a7f3a-2886-4ece-90d3-16a5bb169416 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:50.314252 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a7f3a-2886-4ece-90d3-16a5bb169416 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:50.315067 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519a7f3a-2886-4ece-90d3-16a5bb169416 None admin] RBAC: Adding query filter params (role.id=d0a6f7c5c34a4925b9efb7dd4ac915fc, scope.project.id=6606f4e0816344eca4228a4e9d0ed52e, user.id=4f7f313cdae247c9898ae49d2bfa6a35) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:50.315380 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519a7f3a-2886-4ece-90d3-16a5bb169416 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:50.316902 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519a7f3a-2886-4ece-90d3-16a5bb169416 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:50.328636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 153/294] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 13:59:50 2022] GET /identity/v3/role_assignments?user.id=4f7f313cdae247c9898ae49d2bfa6a35&scope.project.id=6606f4e0816344eca4228a4e9d0ed52e&role.id=d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:50.460421 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b7b27-f11d-4b56-9b20-04e4074e8bd2 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:50.460959 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b7b27-f11d-4b56-9b20-04e4074e8bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:50.460959 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b7b27-f11d-4b56-9b20-04e4074e8bd2 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:50.462302 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 142/295] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:50.469857 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd973e-ccf8-49c4-ac73-679b7a3b9fd9 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:50.470116 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd973e-ccf8-49c4-ac73-679b7a3b9fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:50.470472 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd973e-ccf8-49c4-ac73-679b7a3b9fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:50.473707 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a158c-baa1-4cc3-97c3-3f6f0634ffac None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:50.475010 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a158c-baa1-4cc3-97c3-3f6f0634ffac None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:50.475428 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a158c-baa1-4cc3-97c3-3f6f0634ffac None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:50.476973 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 143/296] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:50.483906 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0495720-876a-44ee-b2b3-5bcdba7b535f None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:50.487093 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0495720-876a-44ee-b2b3-5bcdba7b535f None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:50.487600 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0495720-876a-44ee-b2b3-5bcdba7b535f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:50.496998 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:50.510254 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-2fcd973e-ccf8-49c4-ac73-679b7a3b9fd9 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:50.512014 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-2fcd973e-ccf8-49c4-ac73-679b7a3b9fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:50.514093 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-e0495720-876a-44ee-b2b3-5bcdba7b535f None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:50.518482 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-e0495720-876a-44ee-b2b3-5bcdba7b535f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:50.597093 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 154/297] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:50.598708 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1fa883-9e29-4f55-8a6d-4c31494438dd None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:50.599031 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1fa883-9e29-4f55-8a6d-4c31494438dd None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:50.599309 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1fa883-9e29-4f55-8a6d-4c31494438dd None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:50.600694 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 155/298] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:50.606418 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b3494-4c6d-4535-9430-2bf974cd4216 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:50.606706 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b3494-4c6d-4535-9430-2bf974cd4216 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:50.607084 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b3494-4c6d-4535-9430-2bf974cd4216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:50.625184 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 144/299] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 13:59:50.627243 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-5e8b3494-4c6d-4535-9430-2bf974cd4216 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:50.628451 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-5e8b3494-4c6d-4535-9430-2bf974cd4216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:50.707250 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 156/300] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 13:59:50.887145 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0495720-876a-44ee-b2b3-5bcdba7b535f None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:50.906827 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:50.933971 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b3494-4c6d-4535-9430-2bf974cd4216 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:50.995813 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0495720-876a-44ee-b2b3-5bcdba7b535f None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:51.014378 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:51.034731 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26036ae-247f-4191-a028-3f8a11587059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:51.035764 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26036ae-247f-4191-a028-3f8a11587059 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:51.036046 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26036ae-247f-4191-a028-3f8a11587059 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:51.036321 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26036ae-247f-4191-a028-3f8a11587059 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:51.046890 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26036ae-247f-4191-a028-3f8a11587059 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:51.051691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-d26036ae-247f-4191-a028-3f8a11587059 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 13:59:51.051691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:51.051691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:51.051691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:51.051691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:51.051691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:51.051691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:51.051691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:51.051691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:51.051691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:51.051691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:51.051691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:51.051691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:51.051691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:51.051691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:51.051691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:51.052551 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:51.052551 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:51.052551 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:51.052551 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:51.052551 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:51.052551 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:51.052551 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:51.052551 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:51.052551 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:51.052551 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:51.052551 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:51.052551 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:51.052551 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:51.052551 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:51.052551 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:51.052551 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:51.053334 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:51.053334 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:51.053334 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:51.053334 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:51.053334 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:51.053334 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:51.053334 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:51.053334 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:51.053334 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:51.053334 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:51.053334 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 13:59:51.053334 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 13:59:51.053334 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 145/301] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 13:59:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 13:59:51.053979 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26036ae-247f-4191-a028-3f8a11587059 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:51.060364 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b3494-4c6d-4535-9430-2bf974cd4216 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:51.094321 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e292db41-464a-4cb7-848e-2033d404d0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:51.094904 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292db41-464a-4cb7-848e-2033d404d0d9 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:51.095180 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292db41-464a-4cb7-848e-2033d404d0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:51.095451 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292db41-464a-4cb7-848e-2033d404d0d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:51.096542 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e292db41-464a-4cb7-848e-2033d404d0d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:51.097971 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e292db41-464a-4cb7-848e-2033d404d0d9 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:51.099805 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 157/302] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 13:59:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 13:59:51.100461 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e292db41-464a-4cb7-848e-2033d404d0d9 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:51.144003 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:51.167990 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26036ae-247f-4191-a028-3f8a11587059 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:51.202781 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763ad3f-d379-4460-ada5-434a147b180c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:51.203446 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763ad3f-d379-4460-ada5-434a147b180c None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:51.203825 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763ad3f-d379-4460-ada5-434a147b180c None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:51.204095 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763ad3f-d379-4460-ada5-434a147b180c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:51.209818 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e292db41-464a-4cb7-848e-2033d404d0d9 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:51.215014 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7763ad3f-d379-4460-ada5-434a147b180c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:51.220137 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-7763ad3f-d379-4460-ada5-434a147b180c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 13:59:51.220137 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:51.220137 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:51.220137 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:51.220137 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:51.220137 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:51.220137 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:51.220137 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:51.220137 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:51.220137 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:51.220137 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:51.220137 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:51.220137 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:51.220137 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:51.220137 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:51.220137 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:51.221022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:51.221022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:51.221022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:51.221022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:51.221022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:51.221022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:51.221022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:51.221022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:51.221022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:51.221022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:51.221022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:51.221022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:51.221022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:51.221022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:51.221022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:51.221022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:51.221805 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:51.221805 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:51.221805 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:51.221805 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:51.221805 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:51.221805 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:51.221805 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:51.221805 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:51.221805 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:51.221805 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:51.221805 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 13:59:51.221805 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 13:59:51.221805 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 146/303] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 13:59:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 13:59:51.222673 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763ad3f-d379-4460-ada5-434a147b180c None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:51.245066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:51.251104 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e0c812-7939-49d6-8525-aff863a9fc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:51.251708 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0c812-7939-49d6-8525-aff863a9fc8a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:51.252152 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0c812-7939-49d6-8525-aff863a9fc8a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:51.252426 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0c812-7939-49d6-8525-aff863a9fc8a None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:51.253421 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e0c812-7939-49d6-8525-aff863a9fc8a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:51.253727 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e0c812-7939-49d6-8525-aff863a9fc8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:51.255108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e0c812-7939-49d6-8525-aff863a9fc8a None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:51.263645 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 158/304] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 12 13:59:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:51.268832 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e0c812-7939-49d6-8525-aff863a9fc8a None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:51.329204 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763ad3f-d379-4460-ada5-434a147b180c None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:51.364041 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4bb8de-433c-40b6-956f-d4c0a6124d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:51.364566 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4bb8de-433c-40b6-956f-d4c0a6124d86 None admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:51.364845 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4bb8de-433c-40b6-956f-d4c0a6124d86 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:51.365114 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4bb8de-433c-40b6-956f-d4c0a6124d86 None admin] PATH_INFO: `/v3/groups` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:51.365740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4bb8de-433c-40b6-956f-d4c0a6124d86 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:51.367203 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4bb8de-433c-40b6-956f-d4c0a6124d86 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:51.373878 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e0c812-7939-49d6-8525-aff863a9fc8a None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:51.379854 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:51.391197 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 147/305] 10.222.0.42 () {60 vars in 1134 bytes} [Mon Sep 12 13:59:51 2022] POST /identity/v3/groups => generated 229 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 13:59:51.393510 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4bb8de-433c-40b6-956f-d4c0a6124d86 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:51.419700 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1037fe-4f7c-474a-ba6f-68d5eeac410e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:51.420321 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1037fe-4f7c-474a-ba6f-68d5eeac410e None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:51.420601 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1037fe-4f7c-474a-ba6f-68d5eeac410e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:51.420914 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1037fe-4f7c-474a-ba6f-68d5eeac410e None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:51.422089 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1037fe-4f7c-474a-ba6f-68d5eeac410e None admin] RBAC: Adding query filter params (name=member) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:51.422089 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1037fe-4f7c-474a-ba6f-68d5eeac410e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:51.423450 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1037fe-4f7c-474a-ba6f-68d5eeac410e None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:51.433382 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 159/306] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 12 13:59:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:51.438597 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1037fe-4f7c-474a-ba6f-68d5eeac410e None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:51.524531 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:51.560481 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4bb8de-433c-40b6-956f-d4c0a6124d86 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:51.588551 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1037fe-4f7c-474a-ba6f-68d5eeac410e None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:51.609871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de5000-3230-4473-9832-b778a98fd8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:51.610807 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de5000-3230-4473-9832-b778a98fd8ee None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:51.611126 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de5000-3230-4473-9832-b778a98fd8ee None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:51.611405 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de5000-3230-4473-9832-b778a98fd8ee None admin] PATH_INFO: `/v3/users/6d19c4833ede4e3e9534b15fe101caa9` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:51.626692 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93de5000-3230-4473-9832-b778a98fd8ee None admin] RBAC: Authorizing `identity:get_user(user_id=6d19c4833ede4e3e9534b15fe101caa9)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:51.628156 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93de5000-3230-4473-9832-b778a98fd8ee None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:51.629985 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 148/307] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 13:59:51 2022] GET /identity/v3/users/6d19c4833ede4e3e9534b15fe101caa9 => generated 293 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:51.630996 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426874d-e8e5-45eb-ac1a-e4cfde9c1666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:51.631644 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426874d-e8e5-45eb-ac1a-e4cfde9c1666 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:51.632490 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426874d-e8e5-45eb-ac1a-e4cfde9c1666 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:51.632490 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426874d-e8e5-45eb-ac1a-e4cfde9c1666 None admin] PATH_INFO: `/v3/users/0f4fc0e080a54b35b44b86553cec9320` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:51.633393 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426874d-e8e5-45eb-ac1a-e4cfde9c1666 None admin] RBAC: Authorizing `identity:get_user(user_id=0f4fc0e080a54b35b44b86553cec9320)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:51.635532 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426874d-e8e5-45eb-ac1a-e4cfde9c1666 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:51.637856 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de5000-3230-4473-9832-b778a98fd8ee None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:51.637984 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 160/308] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 13:59:51 2022] GET /identity/v3/users/0f4fc0e080a54b35b44b86553cec9320 => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 13:59:51.644066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426874d-e8e5-45eb-ac1a-e4cfde9c1666 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:51.705818 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:51.751830 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de5000-3230-4473-9832-b778a98fd8ee None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:51.757439 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426874d-e8e5-45eb-ac1a-e4cfde9c1666 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:51.790231 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:51.793043 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f0218-f540-4dd0-b4ed-37ab2980fc09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:51.793820 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f0218-f540-4dd0-b4ed-37ab2980fc09 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:51.794293 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f0218-f540-4dd0-b4ed-37ab2980fc09 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:51.794690 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f0218-f540-4dd0-b4ed-37ab2980fc09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:51.795882 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f0218-f540-4dd0-b4ed-37ab2980fc09 None admin] RBAC: Adding query filter params (role.id=dab439aae69348a183b07971883cebfb, user.id=6d19c4833ede4e3e9534b15fe101caa9, scope.system=all) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:51.796432 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f0218-f540-4dd0-b4ed-37ab2980fc09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:51.798126 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f0218-f540-4dd0-b4ed-37ab2980fc09 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:51.799375 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371de69-c9f8-4415-a54c-73c5ef1d9a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:51.800389 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371de69-c9f8-4415-a54c-73c5ef1d9a4e None admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:51.800789 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371de69-c9f8-4415-a54c-73c5ef1d9a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:51.801068 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371de69-c9f8-4415-a54c-73c5ef1d9a4e None admin] PATH_INFO: `/v3/system/users/0f4fc0e080a54b35b44b86553cec9320/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:51.802726 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6371de69-c9f8-4415-a54c-73c5ef1d9a4e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0f4fc0e080a54b35b44b86553cec9320, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:51.804672 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6371de69-c9f8-4415-a54c-73c5ef1d9a4e None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:51.805127 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 149/309] 10.222.0.42 () {56 vars in 1308 bytes} [Mon Sep 12 13:59:51 2022] GET /identity/v3/role_assignments?user.id=6d19c4833ede4e3e9534b15fe101caa9&scope.system=all&role.id=dab439aae69348a183b07971883cebfb => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:51.826532 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 161/310] 10.222.0.42 () {58 vars in 1334 bytes} [Mon Sep 12 13:59:51 2022] PUT /identity/v3/system/users/0f4fc0e080a54b35b44b86553cec9320/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 13:59:51.912724 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10952a82-890a-43f4-8c6b-7b24c9af3280 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:51.912901 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10952a82-890a-43f4-8c6b-7b24c9af3280 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:51.913323 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10952a82-890a-43f4-8c6b-7b24c9af3280 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:51.915231 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 150/311] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:51.925534 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb569cb-81e8-4663-81a5-9da23d811b88 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:51.925877 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb569cb-81e8-4663-81a5-9da23d811b88 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:51.926139 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb569cb-81e8-4663-81a5-9da23d811b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:51.949280 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-7cb569cb-81e8-4663-81a5-9da23d811b88 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:51.950622 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7cb569cb-81e8-4663-81a5-9da23d811b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:52.033287 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 162/312] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:52.310320 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10952a82-890a-43f4-8c6b-7b24c9af3280 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:52.418522 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10952a82-890a-43f4-8c6b-7b24c9af3280 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:52.439215 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:52.454087 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b0d184-ef1d-4761-9b55-6ddd6debca89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:52.454716 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0d184-ef1d-4761-9b55-6ddd6debca89 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:52.455026 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0d184-ef1d-4761-9b55-6ddd6debca89 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:52.455476 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0d184-ef1d-4761-9b55-6ddd6debca89 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:52.466454 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b0d184-ef1d-4761-9b55-6ddd6debca89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:52.471455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-c6b0d184-ef1d-4761-9b55-6ddd6debca89 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 13:59:52.471455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:52.471455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:52.471455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:52.471455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:52.471455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:52.471455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:52.471455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:52.471455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:52.471455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:52.471455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:52.471455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:52.471455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:52.471455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:52.471455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:52.471455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:52.473456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:52.473456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:52.473456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:52.473456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:52.473456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:52.473456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:52.473456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:52.473456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:52.473456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:52.473456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:52.473456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:52.473456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:52.473456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:52.473456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:52.473456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:52.473456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:52.474926 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:52.474926 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:52.474926 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:52.474926 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:52.474926 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:52.474926 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:52.474926 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:52.474926 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:52.474926 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:52.474926 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:52.474926 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 13:59:52.474926 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 13:59:52.474926 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 151/313] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 13:59:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 13:59:52.478555 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb569cb-81e8-4663-81a5-9da23d811b88 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:52.578068 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb569cb-81e8-4663-81a5-9da23d811b88 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:52.596113 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:52.614386 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6acbd69-a44b-4183-b093-5a969138de0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:52.616419 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6acbd69-a44b-4183-b093-5a969138de0f None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:52.616419 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6acbd69-a44b-4183-b093-5a969138de0f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:52.616419 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6acbd69-a44b-4183-b093-5a969138de0f None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:52.616419 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6acbd69-a44b-4183-b093-5a969138de0f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:52.617432 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6acbd69-a44b-4183-b093-5a969138de0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:52.618245 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6acbd69-a44b-4183-b093-5a969138de0f None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:52.627825 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 163/314] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 12 13:59:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:52.636117 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b0d184-ef1d-4761-9b55-6ddd6debca89 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:52.736149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b0d184-ef1d-4761-9b55-6ddd6debca89 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:52.778017 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbdd0d9-a4df-4e96-921f-3b5edeef6458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:52.778681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdd0d9-a4df-4e96-921f-3b5edeef6458 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:52.778989 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdd0d9-a4df-4e96-921f-3b5edeef6458 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:52.779255 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdd0d9-a4df-4e96-921f-3b5edeef6458 None admin] PATH_INFO: `/v3/users/3b71b8fbb18747eb87004d1157273501` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:52.780549 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbdd0d9-a4df-4e96-921f-3b5edeef6458 None admin] RBAC: Authorizing `identity:get_user(user_id=3b71b8fbb18747eb87004d1157273501)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:52.782202 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbdd0d9-a4df-4e96-921f-3b5edeef6458 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:52.783994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 152/315] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 13:59:52 2022] GET /identity/v3/users/3b71b8fbb18747eb87004d1157273501 => generated 297 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:52.790048 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6acbd69-a44b-4183-b093-5a969138de0f None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:52.893392 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6acbd69-a44b-4183-b093-5a969138de0f None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:52.927069 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490cdfd7-049d-4e24-be65-e0f27a316722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:52.927671 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490cdfd7-049d-4e24-be65-e0f27a316722 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:52.928007 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490cdfd7-049d-4e24-be65-e0f27a316722 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:52.928274 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490cdfd7-049d-4e24-be65-e0f27a316722 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:52.976600 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:52.997063 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490cdfd7-049d-4e24-be65-e0f27a316722 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:52.998498 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490cdfd7-049d-4e24-be65-e0f27a316722 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:53.096861 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 164/316] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 13:59:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 13:59:53.102423 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbdd0d9-a4df-4e96-921f-3b5edeef6458 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:53.158546 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:53.212002 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbdd0d9-a4df-4e96-921f-3b5edeef6458 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:53.249868 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae25b34c-ddd6-41e4-9f7b-4304199d0450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:53.250906 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25b34c-ddd6-41e4-9f7b-4304199d0450 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:53.251394 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25b34c-ddd6-41e4-9f7b-4304199d0450 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:53.251878 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25b34c-ddd6-41e4-9f7b-4304199d0450 None admin] PATH_INFO: `/v3/projects/6606f4e0816344eca4228a4e9d0ed52e` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:53.253226 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae25b34c-ddd6-41e4-9f7b-4304199d0450 None admin] RBAC: Authorizing `identity:get_project(project_id=6606f4e0816344eca4228a4e9d0ed52e)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:53.254905 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae25b34c-ddd6-41e4-9f7b-4304199d0450 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:53.256743 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 153/317] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 13:59:53 2022] GET /identity/v3/projects/6606f4e0816344eca4228a4e9d0ed52e => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:53.262592 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490cdfd7-049d-4e24-be65-e0f27a316722 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:53.347849 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:53.368022 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490cdfd7-049d-4e24-be65-e0f27a316722 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:53.402355 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82991500-534e-4051-9686-97ad121f1111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:53.402999 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82991500-534e-4051-9686-97ad121f1111 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:53.403281 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82991500-534e-4051-9686-97ad121f1111 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:53.403552 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82991500-534e-4051-9686-97ad121f1111 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:53.404274 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82991500-534e-4051-9686-97ad121f1111 None admin] RBAC: Adding query filter params (role.id=dab439aae69348a183b07971883cebfb, scope.project.id=6606f4e0816344eca4228a4e9d0ed52e, user.id=3b71b8fbb18747eb87004d1157273501) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:53.404631 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82991500-534e-4051-9686-97ad121f1111 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:53.406046 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82991500-534e-4051-9686-97ad121f1111 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:53.411879 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 165/318] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 13:59:53 2022] GET /identity/v3/role_assignments?user.id=3b71b8fbb18747eb87004d1157273501&scope.project.id=6606f4e0816344eca4228a4e9d0ed52e&role.id=dab439aae69348a183b07971883cebfb => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:53.737253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced85b7c-5b2b-4319-bf67-94a9b0348cd1 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:53.737755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced85b7c-5b2b-4319-bf67-94a9b0348cd1 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:53.737755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced85b7c-5b2b-4319-bf67-94a9b0348cd1 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:53.739208 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 154/319] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:53.744326 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320d722-6462-49b8-aab0-c7337a84252b None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:53.744611 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320d722-6462-49b8-aab0-c7337a84252b None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:53.744882 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320d722-6462-49b8-aab0-c7337a84252b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:53.763141 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-e320d722-6462-49b8-aab0-c7337a84252b None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:53.764306 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e320d722-6462-49b8-aab0-c7337a84252b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:53.837031 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 166/320] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:54.085766 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de97b78-d5d9-4322-a552-3b83d2ca5f9c None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:54.086134 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de97b78-d5d9-4322-a552-3b83d2ca5f9c None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:54.086310 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de97b78-d5d9-4322-a552-3b83d2ca5f9c None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:54.087830 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 155/321] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:54.092771 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236c81a-8a52-42e0-b7c0-eb6837f4a1f7 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:54.093202 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236c81a-8a52-42e0-b7c0-eb6837f4a1f7 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:54.093373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236c81a-8a52-42e0-b7c0-eb6837f4a1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:54.100147 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf936b-8d67-4f74-9667-0848e414eb30 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:54.100440 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf936b-8d67-4f74-9667-0848e414eb30 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:54.100723 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf936b-8d67-4f74-9667-0848e414eb30 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:54.102026 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 156/322] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 13:59:54.106079 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf936b-8d67-4f74-9667-0848e414eb30 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:54.112669 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-1236c81a-8a52-42e0-b7c0-eb6837f4a1f7 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:54.113858 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1236c81a-8a52-42e0-b7c0-eb6837f4a1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:54.179902 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 167/323] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:54.181249 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b257378-8438-4b5c-89e9-3c4d4a12892b None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:54.182203 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b257378-8438-4b5c-89e9-3c4d4a12892b None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:54.182484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b257378-8438-4b5c-89e9-3c4d4a12892b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:54.193926 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:54.208760 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-1b257378-8438-4b5c-89e9-3c4d4a12892b None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:54.210217 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1b257378-8438-4b5c-89e9-3c4d4a12892b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:54.211930 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf936b-8d67-4f74-9667-0848e414eb30 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:54.228440 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:54.247094 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399380ba-ba74-4a8f-a7d7-c9d79096e974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:54.247721 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399380ba-ba74-4a8f-a7d7-c9d79096e974 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:54.248053 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399380ba-ba74-4a8f-a7d7-c9d79096e974 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:54.248323 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399380ba-ba74-4a8f-a7d7-c9d79096e974 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:54.249437 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399380ba-ba74-4a8f-a7d7-c9d79096e974 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:54.250887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399380ba-ba74-4a8f-a7d7-c9d79096e974 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:54.252445 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 157/324] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 13:59:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 13:59:54.257816 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399380ba-ba74-4a8f-a7d7-c9d79096e974 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:54.287015 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 168/325] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:54.360780 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399380ba-ba74-4a8f-a7d7-c9d79096e974 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:54.394981 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd3eaf-abd9-4b29-b79a-544a5af1ffc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:54.395582 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd3eaf-abd9-4b29-b79a-544a5af1ffc1 None admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:54.395860 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd3eaf-abd9-4b29-b79a-544a5af1ffc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:54.396131 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd3eaf-abd9-4b29-b79a-544a5af1ffc1 None admin] PATH_INFO: `/v3/groups` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:54.396756 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dd3eaf-abd9-4b29-b79a-544a5af1ffc1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:54.398177 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dd3eaf-abd9-4b29-b79a-544a5af1ffc1 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:54.418872 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 158/326] 10.222.0.42 () {60 vars in 1134 bytes} [Mon Sep 12 13:59:54 2022] POST /identity/v3/groups => generated 226 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 13:59:54.447966 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b257378-8438-4b5c-89e9-3c4d4a12892b None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:54.554279 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b257378-8438-4b5c-89e9-3c4d4a12892b None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:54.557732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd3eaf-abd9-4b29-b79a-544a5af1ffc1 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:54.594949 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9860c15-5c5e-4482-9da4-3e71dab661bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:54.595589 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9860c15-5c5e-4482-9da4-3e71dab661bc None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:54.595866 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9860c15-5c5e-4482-9da4-3e71dab661bc None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:54.596135 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9860c15-5c5e-4482-9da4-3e71dab661bc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:54.598010 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:54.609947 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9860c15-5c5e-4482-9da4-3e71dab661bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:54.612936 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:54.614929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-f9860c15-5c5e-4482-9da4-3e71dab661bc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 13:59:54.614929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:54.614929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:54.614929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:54.614929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:54.614929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:54.614929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:54.614929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:54.614929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:54.614929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:54.614929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:54.614929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:54.614929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:54.614929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:54.614929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:54.614929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:54.615799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:54.615799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:54.615799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:54.615799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:54.615799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:54.615799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:54.615799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:54.615799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:54.615799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:54.615799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:54.615799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:54.615799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:54.615799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:54.615799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:54.615799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:54.615799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:54.616598 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:54.616598 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:54.616598 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:54.616598 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:54.616598 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:54.616598 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:54.616598 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:54.616598 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:54.616598 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:54.616598 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:54.616598 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 13:59:54.616598 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 13:59:54.616598 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 169/327] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 13:59:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 13:59:54.622568 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9860c15-5c5e-4482-9da4-3e71dab661bc None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:54.675352 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd3eaf-abd9-4b29-b79a-544a5af1ffc1 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:54.721947 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36efe6f-f0f3-46ab-a273-cc16c1babe49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:54.722960 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36efe6f-f0f3-46ab-a273-cc16c1babe49 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:54.723241 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36efe6f-f0f3-46ab-a273-cc16c1babe49 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:54.723510 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36efe6f-f0f3-46ab-a273-cc16c1babe49 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:54.735810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36efe6f-f0f3-46ab-a273-cc16c1babe49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:54.739202 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9860c15-5c5e-4482-9da4-3e71dab661bc None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:54.741261 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-b36efe6f-f0f3-46ab-a273-cc16c1babe49 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 13:59:54.741261 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:54.741261 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:54.741261 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:54.741261 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:54.741261 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:54.741261 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:54.741261 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:54.741261 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:54.741261 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:54.741261 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:54.741261 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:54.741261 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:54.741261 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:54.741261 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:54.741261 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:54.742432 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:54.742432 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:54.742432 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:54.742432 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:54.742432 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:54.742432 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:54.742432 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:54.742432 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:54.742432 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:54.742432 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:54.742432 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:54.742432 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:54.742432 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:54.742432 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:54.742432 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:54.742432 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:54.743281 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:54.743281 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:54.743281 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:54.743281 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:54.743281 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:54.743281 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:54.743281 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:54.743281 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:54.743281 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:54.743281 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:54.743281 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 13:59:54.743281 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 13:59:54.743281 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 159/328] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 13:59:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 13:59:54.748661 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36efe6f-f0f3-46ab-a273-cc16c1babe49 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:54.782632 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3bc256-9c55-47e0-bc6d-1bade99ed02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:54.783326 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3bc256-9c55-47e0-bc6d-1bade99ed02e None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:54.783610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3bc256-9c55-47e0-bc6d-1bade99ed02e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:54.783881 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3bc256-9c55-47e0-bc6d-1bade99ed02e None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:54.784553 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3bc256-9c55-47e0-bc6d-1bade99ed02e None admin] RBAC: Adding query filter params (name=member) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:54.784851 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3bc256-9c55-47e0-bc6d-1bade99ed02e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:54.786219 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3bc256-9c55-47e0-bc6d-1bade99ed02e None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:54.795598 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 170/329] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 12 13:59:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 13:59:54.803463 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3bc256-9c55-47e0-bc6d-1bade99ed02e None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:54.871495 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:54.884387 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36efe6f-f0f3-46ab-a273-cc16c1babe49 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:54.887515 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:54.933023 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041df6a2-774d-4147-a977-a0df8faa0075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:54.933634 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041df6a2-774d-4147-a977-a0df8faa0075 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:54.933923 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041df6a2-774d-4147-a977-a0df8faa0075 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:54.934184 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041df6a2-774d-4147-a977-a0df8faa0075 None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:54.934292 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3bc256-9c55-47e0-bc6d-1bade99ed02e None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:54.934898 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041df6a2-774d-4147-a977-a0df8faa0075 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:54.935200 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041df6a2-774d-4147-a977-a0df8faa0075 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:54.936634 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041df6a2-774d-4147-a977-a0df8faa0075 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:54.946248 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 160/330] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 12 13:59:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 13:59:54.952124 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041df6a2-774d-4147-a977-a0df8faa0075 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:54.976996 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae807b59-74bf-486d-ad24-643f03dc194b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:54.977700 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae807b59-74bf-486d-ad24-643f03dc194b None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:54.978347 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae807b59-74bf-486d-ad24-643f03dc194b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:54.978621 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae807b59-74bf-486d-ad24-643f03dc194b None admin] PATH_INFO: `/v3/users/0f4fc0e080a54b35b44b86553cec9320` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:54.979799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae807b59-74bf-486d-ad24-643f03dc194b None admin] RBAC: Authorizing `identity:get_user(user_id=0f4fc0e080a54b35b44b86553cec9320)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:54.981218 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae807b59-74bf-486d-ad24-643f03dc194b None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:54.986662 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 171/331] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 13:59:54 2022] GET /identity/v3/users/0f4fc0e080a54b35b44b86553cec9320 => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 13:59:54.991189 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae807b59-74bf-486d-ad24-643f03dc194b None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:55.072858 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041df6a2-774d-4147-a977-a0df8faa0075 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:55.113856 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e9d54-6ba1-4e56-8b81-8ae5c83d5c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:55.114492 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e9d54-6ba1-4e56-8b81-8ae5c83d5c13 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:55.114904 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e9d54-6ba1-4e56-8b81-8ae5c83d5c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:55.115185 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e9d54-6ba1-4e56-8b81-8ae5c83d5c13 None admin] PATH_INFO: `/v3/users/6d19c4833ede4e3e9534b15fe101caa9` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:55.116427 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62e9d54-6ba1-4e56-8b81-8ae5c83d5c13 None admin] RBAC: Authorizing `identity:get_user(user_id=6d19c4833ede4e3e9534b15fe101caa9)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:55.117858 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62e9d54-6ba1-4e56-8b81-8ae5c83d5c13 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:55.118373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae807b59-74bf-486d-ad24-643f03dc194b None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:55.120196 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 161/332] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 13:59:54 2022] GET /identity/v3/users/6d19c4833ede4e3e9534b15fe101caa9 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 13:59:55.127479 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e9d54-6ba1-4e56-8b81-8ae5c83d5c13 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:55.149185 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:55.161053 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75c3b8-c7df-427c-85f7-852faa800405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:55.161661 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75c3b8-c7df-427c-85f7-852faa800405 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:55.161937 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75c3b8-c7df-427c-85f7-852faa800405 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:55.162204 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75c3b8-c7df-427c-85f7-852faa800405 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:55.163016 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb75c3b8-c7df-427c-85f7-852faa800405 None admin] RBAC: Adding query filter params (role.id=d0a6f7c5c34a4925b9efb7dd4ac915fc, user.id=0f4fc0e080a54b35b44b86553cec9320, scope.system=all) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:55.163324 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb75c3b8-c7df-427c-85f7-852faa800405 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:55.164760 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb75c3b8-c7df-427c-85f7-852faa800405 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:55.178619 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 172/333] 10.222.0.42 () {56 vars in 1308 bytes} [Mon Sep 12 13:59:54 2022] GET /identity/v3/role_assignments?user.id=0f4fc0e080a54b35b44b86553cec9320&scope.system=all&role.id=d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 510 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 13:59:55.182773 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:55.254335 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e9d54-6ba1-4e56-8b81-8ae5c83d5c13 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:55.292858 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c828335d-a1b4-4ef6-9bc0-8e9d0bad9fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:55.293515 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828335d-a1b4-4ef6-9bc0-8e9d0bad9fc0 None admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:55.293786 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828335d-a1b4-4ef6-9bc0-8e9d0bad9fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:55.294054 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828335d-a1b4-4ef6-9bc0-8e9d0bad9fc0 None admin] PATH_INFO: `/v3/system/users/6d19c4833ede4e3e9534b15fe101caa9/roles/dab439aae69348a183b07971883cebfb` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:55.295570 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c828335d-a1b4-4ef6-9bc0-8e9d0bad9fc0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6d19c4833ede4e3e9534b15fe101caa9, role_id=dab439aae69348a183b07971883cebfb)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:55.296997 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c828335d-a1b4-4ef6-9bc0-8e9d0bad9fc0 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:55.309345 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 162/334] 10.222.0.42 () {58 vars in 1334 bytes} [Mon Sep 12 13:59:55 2022] PUT /identity/v3/system/users/6d19c4833ede4e3e9534b15fe101caa9/roles/dab439aae69348a183b07971883cebfb => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 13:59:57.137575 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681b399-2f4e-439b-816c-08dffc7e3e38 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:57.138460 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681b399-2f4e-439b-816c-08dffc7e3e38 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:57.138931 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681b399-2f4e-439b-816c-08dffc7e3e38 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:57.140463 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 173/335] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:57.146079 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab085bb6-5bf1-421c-b481-3f8b3449ffa8 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:57.146914 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab085bb6-5bf1-421c-b481-3f8b3449ffa8 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:57.147382 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab085bb6-5bf1-421c-b481-3f8b3449ffa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:57.151305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfdf9b0-8e15-40b2-a7a0-56b2fb7ff533 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:57.151774 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfdf9b0-8e15-40b2-a7a0-56b2fb7ff533 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:57.152215 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfdf9b0-8e15-40b2-a7a0-56b2fb7ff533 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:57.153668 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 174/336] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 13:59:57.159027 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81213521-6b27-4bd4-8ed0-6e565c01f891 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:57.159678 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81213521-6b27-4bd4-8ed0-6e565c01f891 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:57.160162 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81213521-6b27-4bd4-8ed0-6e565c01f891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:57.179308 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-81213521-6b27-4bd4-8ed0-6e565c01f891 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:57.179490 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-ab085bb6-5bf1-421c-b481-3f8b3449ffa8 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:57.180569 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-ab085bb6-5bf1-421c-b481-3f8b3449ffa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:57.181130 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-81213521-6b27-4bd4-8ed0-6e565c01f891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:57.334694 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 175/337] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 13:59:57.336270 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 163/338] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:57.624323 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81213521-6b27-4bd4-8ed0-6e565c01f891 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:57.746228 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81213521-6b27-4bd4-8ed0-6e565c01f891 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:57.760883 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:57.788544 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd5503e-4e8e-4db1-a049-66f05098dd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:57.789190 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd5503e-4e8e-4db1-a049-66f05098dd5a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:57.789474 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd5503e-4e8e-4db1-a049-66f05098dd5a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:57.789752 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd5503e-4e8e-4db1-a049-66f05098dd5a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:57.808400 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd5503e-4e8e-4db1-a049-66f05098dd5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:57.815553 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-fbd5503e-4e8e-4db1-a049-66f05098dd5a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 13:59:57.815553 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:57.815553 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:57.815553 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:57.815553 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:57.815553 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:57.815553 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:57.815553 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:57.815553 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:57.815553 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:57.815553 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:57.815553 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:57.815553 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:57.815553 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:57.815553 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:57.815553 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:57.816476 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:57.816476 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:57.816476 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:57.816476 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:57.816476 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:57.816476 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:57.816476 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:57.816476 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:57.816476 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:57.816476 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:57.816476 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:57.816476 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:57.816476 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:57.816476 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:57.816476 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:57.816476 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:57.817255 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:57.817255 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:57.817255 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:57.817255 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:57.817255 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:57.817255 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:57.817255 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:57.817255 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:57.817255 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:57.817255 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:57.817255 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 13:59:57.817255 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 13:59:57.820322 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 176/339] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 12 13:59:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 13:59:57.827096 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab085bb6-5bf1-421c-b481-3f8b3449ffa8 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:57.870140 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:57.963126 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab085bb6-5bf1-421c-b481-3f8b3449ffa8 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:58.002619 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd5503e-4e8e-4db1-a049-66f05098dd5a None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:58.013368 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d841a40-d41b-46ec-aeed-e67f39771e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:58.013993 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d841a40-d41b-46ec-aeed-e67f39771e1b None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:58.014273 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d841a40-d41b-46ec-aeed-e67f39771e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:58.014544 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d841a40-d41b-46ec-aeed-e67f39771e1b None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:58.015256 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d841a40-d41b-46ec-aeed-e67f39771e1b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:58.015555 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d841a40-d41b-46ec-aeed-e67f39771e1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:58.016957 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d841a40-d41b-46ec-aeed-e67f39771e1b None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:58.036683 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 164/340] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 13:59:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:58.042424 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d841a40-d41b-46ec-aeed-e67f39771e1b None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:58.192967 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd5503e-4e8e-4db1-a049-66f05098dd5a None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:58.198964 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:58.218059 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d841a40-d41b-46ec-aeed-e67f39771e1b None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:58.257383 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15583c8d-ebcb-42cf-8699-3c72f280294c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:58.258033 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15583c8d-ebcb-42cf-8699-3c72f280294c None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:58.258314 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15583c8d-ebcb-42cf-8699-3c72f280294c None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:58.258580 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15583c8d-ebcb-42cf-8699-3c72f280294c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:58.271542 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15583c8d-ebcb-42cf-8699-3c72f280294c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:58.274303 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da71b9-6102-4abc-bf01-c53e964e1006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:58.275140 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da71b9-6102-4abc-bf01-c53e964e1006 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:58.275558 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da71b9-6102-4abc-bf01-c53e964e1006 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:58.275969 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da71b9-6102-4abc-bf01-c53e964e1006 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:58.276992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-15583c8d-ebcb-42cf-8699-3c72f280294c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 13:59:58.276992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:58.276992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:58.276992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:58.276992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:58.276992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:58.276992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:58.276992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:58.276992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:58.276992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:58.276992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:58.276992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:58.276992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:58.276992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:58.276992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:58.276992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:58.278077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:58.278077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:58.278077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:58.278077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:58.278077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:58.278077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:58.278077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:58.278077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:58.278077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:58.278077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:58.278077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:58.278077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:58.278077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:58.278077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:58.278077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:58.278077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:58.279991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:58.279991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:58.279991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:58.279991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:58.279991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:58.279991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:58.279991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:58.279991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:58.279991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:58.279991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:58.279991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 13:59:58.279991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 13:59:58.282285 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 177/341] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 13:59:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 13:59:58.283676 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca20ae0-12cb-4e54-a54f-001b92fe3ebd None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:58.284168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca20ae0-12cb-4e54-a54f-001b92fe3ebd None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:58.284621 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca20ae0-12cb-4e54-a54f-001b92fe3ebd None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:58.286644 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 178/342] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:58.288474 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682a479-3e7c-4c1b-bcf7-2a77bb03f482 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:58.288939 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682a479-3e7c-4c1b-bcf7-2a77bb03f482 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:58.289383 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682a479-3e7c-4c1b-bcf7-2a77bb03f482 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:58.291606 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 179/343] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:58.292515 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682a479-3e7c-4c1b-bcf7-2a77bb03f482 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:58.329535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:58.364569 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da71b9-6102-4abc-bf01-c53e964e1006 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:58.366109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da71b9-6102-4abc-bf01-c53e964e1006 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:58.422634 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682a479-3e7c-4c1b-bcf7-2a77bb03f482 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:58.471299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a015ec4-c038-4867-8fc7-02014b8bc242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:58.471834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a015ec4-c038-4867-8fc7-02014b8bc242 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:58.472251 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a015ec4-c038-4867-8fc7-02014b8bc242 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:58.472575 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a015ec4-c038-4867-8fc7-02014b8bc242 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:58.473319 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a015ec4-c038-4867-8fc7-02014b8bc242 None admin] RBAC: Adding query filter params (name=member) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:58.473619 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a015ec4-c038-4867-8fc7-02014b8bc242 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:58.474155 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:58.475293 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a015ec4-c038-4867-8fc7-02014b8bc242 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:58.485952 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 180/344] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 12 13:59:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:58.487221 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119ad68-860f-4887-900f-49ee5d76749a None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:58.487506 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119ad68-860f-4887-900f-49ee5d76749a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:58.487779 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119ad68-860f-4887-900f-49ee5d76749a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:58.488853 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 165/345] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 13:59:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 12 13:59:58.490263 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dcf22c-6f94-41c8-929a-5e124c0ab872 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:58.490727 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dcf22c-6f94-41c8-929a-5e124c0ab872 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:58.495321 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dcf22c-6f94-41c8-929a-5e124c0ab872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:58.514794 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-c119ad68-860f-4887-900f-49ee5d76749a None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:58.516026 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c119ad68-860f-4887-900f-49ee5d76749a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:58.530909 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-92dcf22c-6f94-41c8-929a-5e124c0ab872 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:58.532089 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-92dcf22c-6f94-41c8-929a-5e124c0ab872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:58.604717 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 181/346] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:58.605908 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119ad68-860f-4887-900f-49ee5d76749a None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:58.615171 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 166/347] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:58.620156 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dcf22c-6f94-41c8-929a-5e124c0ab872 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:58.634114 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:58.721085 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119ad68-860f-4887-900f-49ee5d76749a None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:58.765925 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bd1e3c-16a2-42bf-b6ee-31f74664b8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:58.766959 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd1e3c-16a2-42bf-b6ee-31f74664b8d7 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:58.767467 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd1e3c-16a2-42bf-b6ee-31f74664b8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:58.767911 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd1e3c-16a2-42bf-b6ee-31f74664b8d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:58.862264 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bd1e3c-16a2-42bf-b6ee-31f74664b8d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:58.864124 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dcf22c-6f94-41c8-929a-5e124c0ab872 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:58.864407 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bd1e3c-16a2-42bf-b6ee-31f74664b8d7 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:58.891245 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:58.910263 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:58.927027 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f72952-01e1-475a-b895-ed4d911ed57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:58.927664 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f72952-01e1-475a-b895-ed4d911ed57e None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:58.927942 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f72952-01e1-475a-b895-ed4d911ed57e None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:58.928214 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f72952-01e1-475a-b895-ed4d911ed57e None admin] PATH_INFO: `/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:58.929565 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f72952-01e1-475a-b895-ed4d911ed57e None admin] RBAC: Authorizing `identity:get_project(project_id=e10c65c7a17c44a7a8711e7701efdcd0)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:58.930767 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f72952-01e1-475a-b895-ed4d911ed57e None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:58.932996 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 167/348] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 13:59:58 2022] GET /identity/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0 => generated 298 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:58.939839 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f72952-01e1-475a-b895-ed4d911ed57e None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:58.980446 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 182/349] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 13:59:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 13:59:58.985580 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bd1e3c-16a2-42bf-b6ee-31f74664b8d7 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:59.097703 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f72952-01e1-475a-b895-ed4d911ed57e None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:59.124882 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bd1e3c-16a2-42bf-b6ee-31f74664b8d7 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:59.151101 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a771b-a67c-497d-a099-5d6848ea9403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:59.151747 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a771b-a67c-497d-a099-5d6848ea9403 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:59.152027 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a771b-a67c-497d-a099-5d6848ea9403 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:59.152300 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a771b-a67c-497d-a099-5d6848ea9403 None admin] PATH_INFO: `/v3/groups/2378a09b5ad64b1eb9d19d3c3d05296a` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:59.159560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a771b-a67c-497d-a099-5d6848ea9403 None admin] RBAC: Authorizing `identity:get_group(group_id=2378a09b5ad64b1eb9d19d3c3d05296a)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:59.161016 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a771b-a67c-497d-a099-5d6848ea9403 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:59.163657 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 168/350] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Sep 12 13:59:58 2022] GET /identity/v3/groups/2378a09b5ad64b1eb9d19d3c3d05296a => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:59.163788 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:59.163881 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a771b-a67c-497d-a099-5d6848ea9403 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:59.170408 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2253726f-abc5-4e7c-bf0a-9ce76e8cc519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:59.171637 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253726f-abc5-4e7c-bf0a-9ce76e8cc519 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:59.172139 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253726f-abc5-4e7c-bf0a-9ce76e8cc519 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:59.172621 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253726f-abc5-4e7c-bf0a-9ce76e8cc519 None admin] PATH_INFO: `/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:59.173995 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2253726f-abc5-4e7c-bf0a-9ce76e8cc519 None admin] RBAC: Authorizing `identity:get_project(project_id=e10c65c7a17c44a7a8711e7701efdcd0)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:59.176180 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2253726f-abc5-4e7c-bf0a-9ce76e8cc519 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:59.178533 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 183/351] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 13:59:58 2022] GET /identity/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 13:59:59.179553 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2253726f-abc5-4e7c-bf0a-9ce76e8cc519 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:59.243228 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:59.299497 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a771b-a67c-497d-a099-5d6848ea9403 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:59.313795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2253726f-abc5-4e7c-bf0a-9ce76e8cc519 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:59.349724 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0a574-4576-4abf-8fe2-8ce1342032e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:59.351002 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0a574-4576-4abf-8fe2-8ce1342032e8 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:59.351713 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0a574-4576-4abf-8fe2-8ce1342032e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:59.352614 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0a574-4576-4abf-8fe2-8ce1342032e8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:59.357507 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb8d96-708e-4964-91be-f60b6c846b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:59.358115 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb8d96-708e-4964-91be-f60b6c846b62 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:59.358389 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb8d96-708e-4964-91be-f60b6c846b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:59.358655 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb8d96-708e-4964-91be-f60b6c846b62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:59.359444 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bb8d96-708e-4964-91be-f60b6c846b62 None admin] RBAC: Adding query filter params (group.id=2378a09b5ad64b1eb9d19d3c3d05296a, role.id=bcde1b76af8048deb2829c5aaa27296c, scope.project.id=e10c65c7a17c44a7a8711e7701efdcd0) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:59.359745 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bb8d96-708e-4964-91be-f60b6c846b62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:59.362221 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bb8d96-708e-4964-91be-f60b6c846b62 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:59.367444 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f0a574-4576-4abf-8fe2-8ce1342032e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:59.371040 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 184/352] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Sep 12 13:59:59 2022] GET /identity/v3/role_assignments?group.id=2378a09b5ad64b1eb9d19d3c3d05296a&scope.project.id=e10c65c7a17c44a7a8711e7701efdcd0&role.id=bcde1b76af8048deb2829c5aaa27296c => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:59.371967 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb8d96-708e-4964-91be-f60b6c846b62 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:59.373879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-12f0a574-4576-4abf-8fe2-8ce1342032e8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 13:59:59.373879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:59.373879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:59.373879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:59.373879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:59.373879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:59.373879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:59.373879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:59.373879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:59.373879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:59.373879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:59.373879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:59.373879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:59.373879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:59.373879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:59.373879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:59.374828 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:59.374828 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:59.374828 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:59.374828 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:59.374828 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:59.374828 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:59.374828 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:59.374828 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:59.374828 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:59.374828 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:59.374828 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:59.374828 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:59.374828 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:59.374828 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:59.374828 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:59.374828 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:59.375759 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:59.375759 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:59.375759 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:59.375759 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:59.375759 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:59.375759 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:59.375759 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:59.375759 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:59.375759 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:59.375759 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:59.375759 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 13:59:59.375759 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 13:59:59.378709 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 169/353] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 13:59:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 13:59:59.379931 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0a574-4576-4abf-8fe2-8ce1342032e8 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:59.448540 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:59.502142 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0a574-4576-4abf-8fe2-8ce1342032e8 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:59.511185 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb8d96-708e-4964-91be-f60b6c846b62 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:59.517540 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:59.548073 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9a4552-d684-48ac-81a4-cd679103ba12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:59.549727 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a4552-d684-48ac-81a4-cd679103ba12 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:59.550240 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a4552-d684-48ac-81a4-cd679103ba12 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:59.550919 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a4552-d684-48ac-81a4-cd679103ba12 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:59.558985 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4d8ad-7ed7-437d-99ab-0698cc593dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:59.559641 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4d8ad-7ed7-437d-99ab-0698cc593dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:59.559919 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4d8ad-7ed7-437d-99ab-0698cc593dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:59.560188 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4d8ad-7ed7-437d-99ab-0698cc593dd6 None admin] PATH_INFO: `/v3/groups/2378a09b5ad64b1eb9d19d3c3d05296a` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:59.561345 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b4d8ad-7ed7-437d-99ab-0698cc593dd6 None admin] RBAC: Authorizing `identity:get_group(group_id=2378a09b5ad64b1eb9d19d3c3d05296a)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:59.563724 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b4d8ad-7ed7-437d-99ab-0698cc593dd6 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:59.565827 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 185/354] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Sep 12 13:59:59 2022] GET /identity/v3/groups/2378a09b5ad64b1eb9d19d3c3d05296a => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:59.567089 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806e22b-4ec0-4d84-96a4-afd1dfd1b6f6 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:59.567372 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806e22b-4ec0-4d84-96a4-afd1dfd1b6f6 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:59.567644 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806e22b-4ec0-4d84-96a4-afd1dfd1b6f6 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:59.568884 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 186/355] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 13:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 13:59:59.569541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9a4552-d684-48ac-81a4-cd679103ba12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 13:59:59.569658 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806e22b-4ec0-4d84-96a4-afd1dfd1b6f6 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:59.577080 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-6c9a4552-d684-48ac-81a4-cd679103ba12 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 13:59:59.577080 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 13:59:59.577080 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 13:59:59.577080 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 13:59:59.577080 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 13:59:59.577080 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 13:59:59.577080 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 13:59:59.577080 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 13:59:59.577080 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 13:59:59.577080 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 13:59:59.577080 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 13:59:59.577080 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 13:59:59.577080 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 13:59:59.577080 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 13:59:59.577080 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 13:59:59.577080 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 13:59:59.578010 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 13:59:59.578010 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 13:59:59.578010 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 13:59:59.578010 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 13:59:59.578010 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 13:59:59.578010 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 13:59:59.578010 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 13:59:59.578010 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 13:59:59.578010 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 13:59:59.578010 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 13:59:59.578010 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 13:59:59.578010 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 13:59:59.578010 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 13:59:59.578010 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 13:59:59.578010 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 13:59:59.578010 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 13:59:59.578791 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 13:59:59.578791 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 13:59:59.578791 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 13:59:59.578791 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 13:59:59.578791 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 13:59:59.578791 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 13:59:59.578791 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 13:59:59.578791 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 13:59:59.578791 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 13:59:59.578791 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 13:59:59.578791 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 13:59:59.578791 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 13:59:59.581660 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 170/356] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 13:59:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 13:59:59.583239 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9a4552-d684-48ac-81a4-cd679103ba12 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:59.698101 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9a4552-d684-48ac-81a4-cd679103ba12 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:59.702238 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806e22b-4ec0-4d84-96a4-afd1dfd1b6f6 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 13:59:59.731606 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:59.736142 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd0940-2d9c-4096-a2c0-74f96ae46a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:59.737044 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd0940-2d9c-4096-a2c0-74f96ae46a48 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:59.737740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd0940-2d9c-4096-a2c0-74f96ae46a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:59.738231 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd0940-2d9c-4096-a2c0-74f96ae46a48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:59.739209 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cd0940-2d9c-4096-a2c0-74f96ae46a48 None admin] RBAC: Adding query filter params (group.id=2378a09b5ad64b1eb9d19d3c3d05296a, role.id=d0a6f7c5c34a4925b9efb7dd4ac915fc, scope.project.id=e10c65c7a17c44a7a8711e7701efdcd0) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:59.739819 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cd0940-2d9c-4096-a2c0-74f96ae46a48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:59.741531 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cd0940-2d9c-4096-a2c0-74f96ae46a48 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:59.744158 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c6316-1615-4767-a484-925a4cdbb82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 13:59:59.745229 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c6316-1615-4767-a484-925a4cdbb82b None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:59.745603 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c6316-1615-4767-a484-925a4cdbb82b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:59.745968 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c6316-1615-4767-a484-925a4cdbb82b None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:59.747103 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58c6316-1615-4767-a484-925a4cdbb82b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 13:59:59.747595 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58c6316-1615-4767-a484-925a4cdbb82b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 13:59:59.749918 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58c6316-1615-4767-a484-925a4cdbb82b None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 13:59:59.753751 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 171/357] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Sep 12 13:59:59 2022] GET /identity/v3/role_assignments?group.id=2378a09b5ad64b1eb9d19d3c3d05296a&scope.project.id=e10c65c7a17c44a7a8711e7701efdcd0&role.id=d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:59.755588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3bbbf-54e6-4f7e-87fd-f45c63b8b2fb None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 13:59:59.756091 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3bbbf-54e6-4f7e-87fd-f45c63b8b2fb None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 13:59:59.756676 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3bbbf-54e6-4f7e-87fd-f45c63b8b2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 13:59:59.763229 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 187/358] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 12 13:59:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 13:59:59.764121 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c6316-1615-4767-a484-925a4cdbb82b None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:59.783033 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-03a3bbbf-54e6-4f7e-87fd-f45c63b8b2fb None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 13:59:59.784957 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-03a3bbbf-54e6-4f7e-87fd-f45c63b8b2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 13:59:59.827058 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 13:59:59.891139 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 172/359] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 13:59:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 13:59:59.892922 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a3bbbf-54e6-4f7e-87fd-f45c63b8b2fb None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 13:59:59.940193 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c6316-1615-4767-a484-925a4cdbb82b None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:00.014732 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d97ed8c-da59-4709-a3d3-db581fe61d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:00.015435 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97ed8c-da59-4709-a3d3-db581fe61d46 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:00.015711 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97ed8c-da59-4709-a3d3-db581fe61d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:00.015976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97ed8c-da59-4709-a3d3-db581fe61d46 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:00.016648 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d97ed8c-da59-4709-a3d3-db581fe61d46 None admin] RBAC: Adding query filter params (name=member) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:00.016944 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d97ed8c-da59-4709-a3d3-db581fe61d46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:00.018294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d97ed8c-da59-4709-a3d3-db581fe61d46 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:00.031801 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 188/360] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 12 13:59:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:00.035518 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dfb642-6e3c-4485-a85e-b46fb7d6cb5f None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:00.036443 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dfb642-6e3c-4485-a85e-b46fb7d6cb5f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:00.039702 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dfb642-6e3c-4485-a85e-b46fb7d6cb5f None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:00.041187 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 189/361] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:00 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:00.041954 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dfb642-6e3c-4485-a85e-b46fb7d6cb5f None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:00.094185 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a3bbbf-54e6-4f7e-87fd-f45c63b8b2fb None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:00.115370 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:00.136958 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da33036-cb22-404a-88f3-1f0d09bb8708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:00.137955 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da33036-cb22-404a-88f3-1f0d09bb8708 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:00.138446 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da33036-cb22-404a-88f3-1f0d09bb8708 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:00.138974 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da33036-cb22-404a-88f3-1f0d09bb8708 None admin] PATH_INFO: `/v3/users/6d19c4833ede4e3e9534b15fe101caa9` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:00.140339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da33036-cb22-404a-88f3-1f0d09bb8708 None admin] RBAC: Authorizing `identity:get_user(user_id=6d19c4833ede4e3e9534b15fe101caa9)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:00.142956 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da33036-cb22-404a-88f3-1f0d09bb8708 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:00.145066 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 173/362] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 13:59:59 2022] GET /identity/v3/users/6d19c4833ede4e3e9534b15fe101caa9 => generated 293 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:00.146429 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a37622-6be0-4b32-a3d0-5443451f02bc None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:00.146936 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a37622-6be0-4b32-a3d0-5443451f02bc None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:00.147436 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a37622-6be0-4b32-a3d0-5443451f02bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:00.170378 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-c4a37622-6be0-4b32-a3d0-5443451f02bc None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:00.172036 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-c4a37622-6be0-4b32-a3d0-5443451f02bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:00.183922 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dfb642-6e3c-4485-a85e-b46fb7d6cb5f None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:00.208738 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:00.232746 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f3de43-14e6-4e05-bc34-2009006e0869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:00.233766 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3de43-14e6-4e05-bc34-2009006e0869 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:00.234149 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3de43-14e6-4e05-bc34-2009006e0869 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:00.234525 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3de43-14e6-4e05-bc34-2009006e0869 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:00.264266 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 174/363] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:00.266369 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a37622-6be0-4b32-a3d0-5443451f02bc None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:00.346986 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f3de43-14e6-4e05-bc34-2009006e0869 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:00.348488 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f3de43-14e6-4e05-bc34-2009006e0869 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:00.436201 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:00.439319 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a37622-6be0-4b32-a3d0-5443451f02bc None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:00.516253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1df36-d81a-4176-86f8-ceed9db1158b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:00.516939 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1df36-d81a-4176-86f8-ceed9db1158b None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:00.517165 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1df36-d81a-4176-86f8-ceed9db1158b None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:00.517436 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1df36-d81a-4176-86f8-ceed9db1158b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:00.518182 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d1df36-d81a-4176-86f8-ceed9db1158b None admin] RBAC: Adding query filter params (role.id=dab439aae69348a183b07971883cebfb, user.id=6d19c4833ede4e3e9534b15fe101caa9, scope.system=all) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:00.518486 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d1df36-d81a-4176-86f8-ceed9db1158b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:00.519978 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d1df36-d81a-4176-86f8-ceed9db1158b None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:00.522926 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 190/364] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:00:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 14:00:00.523732 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f3de43-14e6-4e05-bc34-2009006e0869 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:00.545887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 175/365] 10.222.0.42 () {56 vars in 1308 bytes} [Mon Sep 12 14:00:00 2022] GET /identity/v3/role_assignments?user.id=6d19c4833ede4e3e9534b15fe101caa9&scope.system=all&role.id=dab439aae69348a183b07971883cebfb => generated 510 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:00.547634 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1df36-d81a-4176-86f8-ceed9db1158b None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:00.598337 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:00.681033 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f3de43-14e6-4e05-bc34-2009006e0869 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:00.703615 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1df36-d81a-4176-86f8-ceed9db1158b None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:00.725782 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadd6ba2-1466-451f-97f1-ba89cba24104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:00.726653 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadd6ba2-1466-451f-97f1-ba89cba24104 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:00.726963 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadd6ba2-1466-451f-97f1-ba89cba24104 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:00.727236 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadd6ba2-1466-451f-97f1-ba89cba24104 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:00.739173 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadd6ba2-1466-451f-97f1-ba89cba24104 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:00.747261 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-dadd6ba2-1466-451f-97f1-ba89cba24104 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 14:00:00.747261 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:00.747261 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:00.747261 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:00.747261 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:00.747261 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:00.747261 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:00.747261 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:00.747261 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:00.747261 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:00.747261 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:00.747261 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:00.747261 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:00.747261 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:00.747261 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:00.747261 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:00.748208 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:00.748208 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:00.748208 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:00.748208 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:00.748208 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:00.748208 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:00.748208 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:00.748208 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:00.748208 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:00.748208 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:00.748208 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:00.748208 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:00.748208 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:00.748208 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:00.748208 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:00.748208 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:00.748989 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:00.748989 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:00.748989 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:00.748989 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:00.748989 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:00.748989 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:00.748989 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:00.748989 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:00.748989 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:00.748989 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:00.748989 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 14:00:00.748989 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:00.752137 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 191/366] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 12 14:00:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:00.753101 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadd6ba2-1466-451f-97f1-ba89cba24104 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:00.753792 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36e671-3a5c-4097-8a6b-095b2b860c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:00.754586 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36e671-3a5c-4097-8a6b-095b2b860c06 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:00.754999 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36e671-3a5c-4097-8a6b-095b2b860c06 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:00.755368 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36e671-3a5c-4097-8a6b-095b2b860c06 None admin] PATH_INFO: `/v3/projects/6606f4e0816344eca4228a4e9d0ed52e` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:00.756620 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba36e671-3a5c-4097-8a6b-095b2b860c06 None admin] RBAC: Authorizing `identity:get_project(project_id=6606f4e0816344eca4228a4e9d0ed52e)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:00.758201 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba36e671-3a5c-4097-8a6b-095b2b860c06 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:00.760546 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 176/367] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 14:00:00 2022] GET /identity/v3/projects/6606f4e0816344eca4228a4e9d0ed52e => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:00.761473 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36e671-3a5c-4097-8a6b-095b2b860c06 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:00.769056 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:00.917186 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadd6ba2-1466-451f-97f1-ba89cba24104 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:00.939412 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:00.955870 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36e671-3a5c-4097-8a6b-095b2b860c06 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:00.964044 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91ef4ce-9034-479c-8cae-c0f8b216d3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:00.965007 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ef4ce-9034-479c-8cae-c0f8b216d3dd None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:00.965482 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ef4ce-9034-479c-8cae-c0f8b216d3dd None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:00.965966 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ef4ce-9034-479c-8cae-c0f8b216d3dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:00.981165 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91ef4ce-9034-479c-8cae-c0f8b216d3dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:00.992100 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-c91ef4ce-9034-479c-8cae-c0f8b216d3dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:00:00.992100 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:00.992100 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:00.992100 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:00.992100 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:00.992100 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:00.992100 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:00.992100 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:00.992100 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:00.992100 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:00.992100 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:00.992100 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:00.992100 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:00.992100 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:00.992100 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:00.992100 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:00.993025 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:00.993025 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:00.993025 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:00.993025 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:00.993025 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:00.993025 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:00.993025 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:00.993025 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:00.993025 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:00.993025 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:00.993025 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:00.993025 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:00.993025 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:00.993025 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:00.993025 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:00.993025 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:00.993839 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:00.993839 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:00.993839 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:00.993839 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:00.993839 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:00.993839 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:00.993839 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:00.993839 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:00.993839 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:00.993839 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:00.993839 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:00:00.993839 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:00.996222 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 192/368] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 12 14:00:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:00.997327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91ef4ce-9034-479c-8cae-c0f8b216d3dd None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:01.014799 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b849a259-fd85-4833-bbfc-b5a701d5c0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:01.015632 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849a259-fd85-4833-bbfc-b5a701d5c0c1 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:01.016009 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849a259-fd85-4833-bbfc-b5a701d5c0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:01.016379 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849a259-fd85-4833-bbfc-b5a701d5c0c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:01.017157 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b849a259-fd85-4833-bbfc-b5a701d5c0c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:01.017562 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b849a259-fd85-4833-bbfc-b5a701d5c0c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:01.019151 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b849a259-fd85-4833-bbfc-b5a701d5c0c1 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:01.031121 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 177/369] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:00:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:01.032120 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b849a259-fd85-4833-bbfc-b5a701d5c0c1 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:01.080425 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:01.118922 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91ef4ce-9034-479c-8cae-c0f8b216d3dd None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:01.152874 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b849a259-fd85-4833-bbfc-b5a701d5c0c1 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:01.164275 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b281b159-e628-418f-b8f1-c9beb784e2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:01.165090 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281b159-e628-418f-b8f1-c9beb784e2dd None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:01.165581 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281b159-e628-418f-b8f1-c9beb784e2dd None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:01.166813 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281b159-e628-418f-b8f1-c9beb784e2dd None admin] PATH_INFO: `/v3/groups/2378a09b5ad64b1eb9d19d3c3d05296a` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:01.167863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b281b159-e628-418f-b8f1-c9beb784e2dd None admin] RBAC: Authorizing `identity:get_group(group_id=2378a09b5ad64b1eb9d19d3c3d05296a)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:01.169405 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b281b159-e628-418f-b8f1-c9beb784e2dd None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:01.172274 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 193/370] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Sep 12 14:00:00 2022] GET /identity/v3/groups/2378a09b5ad64b1eb9d19d3c3d05296a => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:01.173160 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b281b159-e628-418f-b8f1-c9beb784e2dd None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:01.203092 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94edbe0-4b85-4b85-b78c-4edc3acc3abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:01.204575 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94edbe0-4b85-4b85-b78c-4edc3acc3abd None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:01.204965 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94edbe0-4b85-4b85-b78c-4edc3acc3abd None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:01.205669 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94edbe0-4b85-4b85-b78c-4edc3acc3abd None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:01.206509 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94edbe0-4b85-4b85-b78c-4edc3acc3abd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:01.206957 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94edbe0-4b85-4b85-b78c-4edc3acc3abd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:01.209311 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94edbe0-4b85-4b85-b78c-4edc3acc3abd None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:01.223670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 178/371] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 14:00:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:01.224612 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94edbe0-4b85-4b85-b78c-4edc3acc3abd None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:01.245434 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:01.295112 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b281b159-e628-418f-b8f1-c9beb784e2dd None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:01.335224 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4abadd-22c4-402b-a1c2-947d4a0f1e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:01.336173 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4abadd-22c4-402b-a1c2-947d4a0f1e73 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:01.336684 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4abadd-22c4-402b-a1c2-947d4a0f1e73 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:01.337288 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4abadd-22c4-402b-a1c2-947d4a0f1e73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:01.343892 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:01.352590 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94edbe0-4b85-4b85-b78c-4edc3acc3abd None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:01.395817 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ee3c0-bf05-44a9-8e61-a101d3ce3ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:01.396430 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ee3c0-bf05-44a9-8e61-a101d3ce3ba8 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:01.396710 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ee3c0-bf05-44a9-8e61-a101d3ce3ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:01.396983 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ee3c0-bf05-44a9-8e61-a101d3ce3ba8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:01.397821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ee3c0-bf05-44a9-8e61-a101d3ce3ba8 None admin] RBAC: Adding query filter params (group.id=2378a09b5ad64b1eb9d19d3c3d05296a, role.id=d0a6f7c5c34a4925b9efb7dd4ac915fc, scope.project.id=6606f4e0816344eca4228a4e9d0ed52e) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:01.398129 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ee3c0-bf05-44a9-8e61-a101d3ce3ba8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:01.399679 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ee3c0-bf05-44a9-8e61-a101d3ce3ba8 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:01.405990 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 179/372] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Sep 12 14:00:01 2022] GET /identity/v3/role_assignments?group.id=2378a09b5ad64b1eb9d19d3c3d05296a&scope.project.id=6606f4e0816344eca4228a4e9d0ed52e&role.id=d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:01.407047 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ee3c0-bf05-44a9-8e61-a101d3ce3ba8 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:01.420443 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4abadd-22c4-402b-a1c2-947d4a0f1e73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:01.422556 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4abadd-22c4-402b-a1c2-947d4a0f1e73 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:01.539564 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 194/373] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:00:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 14:00:01.548609 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4abadd-22c4-402b-a1c2-947d4a0f1e73 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:01.552297 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:01.599282 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ee3c0-bf05-44a9-8e61-a101d3ce3ba8 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:01.623544 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:01.654164 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9df0c1-52ef-4312-bfe6-b9a0728347f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:01.654871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9df0c1-52ef-4312-bfe6-b9a0728347f0 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:01.655224 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9df0c1-52ef-4312-bfe6-b9a0728347f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:01.655537 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9df0c1-52ef-4312-bfe6-b9a0728347f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:01.718723 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4abadd-22c4-402b-a1c2-947d4a0f1e73 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:01.733865 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9df0c1-52ef-4312-bfe6-b9a0728347f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:01.735408 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9df0c1-52ef-4312-bfe6-b9a0728347f0 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:01.766158 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd73ddf-cff4-4535-8a5b-a8b21de9edb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:01.767151 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd73ddf-cff4-4535-8a5b-a8b21de9edb2 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:01.767572 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd73ddf-cff4-4535-8a5b-a8b21de9edb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:01.767982 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd73ddf-cff4-4535-8a5b-a8b21de9edb2 None admin] PATH_INFO: `/v3/projects/6606f4e0816344eca4228a4e9d0ed52e` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:01.769118 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd73ddf-cff4-4535-8a5b-a8b21de9edb2 None admin] RBAC: Authorizing `identity:get_project(project_id=6606f4e0816344eca4228a4e9d0ed52e)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:01.770559 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd73ddf-cff4-4535-8a5b-a8b21de9edb2 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:01.772528 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 195/374] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 14:00:01 2022] GET /identity/v3/projects/6606f4e0816344eca4228a4e9d0ed52e => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:01.780510 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd73ddf-cff4-4535-8a5b-a8b21de9edb2 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:01.842306 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:01.874865 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 180/375] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:00:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 14:00:01.882081 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9df0c1-52ef-4312-bfe6-b9a0728347f0 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:01.923836 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd73ddf-cff4-4535-8a5b-a8b21de9edb2 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:01.927581 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:01.968947 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be9e3d8-01a7-42ae-839a-4a3a519abbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:01.969599 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9e3d8-01a7-42ae-839a-4a3a519abbd4 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:01.969878 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9e3d8-01a7-42ae-839a-4a3a519abbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:01.970278 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9e3d8-01a7-42ae-839a-4a3a519abbd4 None admin] PATH_INFO: `/v3/groups/2378a09b5ad64b1eb9d19d3c3d05296a` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:01.971527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be9e3d8-01a7-42ae-839a-4a3a519abbd4 None admin] RBAC: Authorizing `identity:get_group(group_id=2378a09b5ad64b1eb9d19d3c3d05296a)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:01.972957 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be9e3d8-01a7-42ae-839a-4a3a519abbd4 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:01.976299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 196/376] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Sep 12 14:00:01 2022] GET /identity/v3/groups/2378a09b5ad64b1eb9d19d3c3d05296a => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:01.981951 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be9e3d8-01a7-42ae-839a-4a3a519abbd4 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:02.030053 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9df0c1-52ef-4312-bfe6-b9a0728347f0 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:02.135027 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be9e3d8-01a7-42ae-839a-4a3a519abbd4 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:02.144106 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944ce592-b0d6-47fa-98e4-41cea1aef1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:02.144753 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ce592-b0d6-47fa-98e4-41cea1aef1e5 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:02.145027 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ce592-b0d6-47fa-98e4-41cea1aef1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:02.145292 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ce592-b0d6-47fa-98e4-41cea1aef1e5 None admin] PATH_INFO: `/v3/projects/b159fcc4f05d45de88c679e235640813` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:02.151554 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944ce592-b0d6-47fa-98e4-41cea1aef1e5 None admin] RBAC: Authorizing `identity:get_project(project_id=b159fcc4f05d45de88c679e235640813)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:02.153274 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944ce592-b0d6-47fa-98e4-41cea1aef1e5 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:02.155281 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 181/377] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 14:00:01 2022] GET /identity/v3/projects/b159fcc4f05d45de88c679e235640813 => generated 344 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:02.169694 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944ce592-b0d6-47fa-98e4-41cea1aef1e5 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:02.182274 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a714d6-c304-44c5-ad5c-ccc9f58c168b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:02.183252 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a714d6-c304-44c5-ad5c-ccc9f58c168b None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:02.183723 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a714d6-c304-44c5-ad5c-ccc9f58c168b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:02.184200 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a714d6-c304-44c5-ad5c-ccc9f58c168b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:02.185252 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a714d6-c304-44c5-ad5c-ccc9f58c168b None admin] RBAC: Adding query filter params (group.id=2378a09b5ad64b1eb9d19d3c3d05296a, role.id=bcde1b76af8048deb2829c5aaa27296c, scope.project.id=6606f4e0816344eca4228a4e9d0ed52e) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:02.185754 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a714d6-c304-44c5-ad5c-ccc9f58c168b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:02.187381 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a714d6-c304-44c5-ad5c-ccc9f58c168b None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:02.193894 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 197/378] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Sep 12 14:00:01 2022] GET /identity/v3/role_assignments?group.id=2378a09b5ad64b1eb9d19d3c3d05296a&scope.project.id=6606f4e0816344eca4228a4e9d0ed52e&role.id=bcde1b76af8048deb2829c5aaa27296c => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:00:02.227496 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a26c78-bdff-4260-915c-6d74f51c9c50 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:02.228738 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a26c78-bdff-4260-915c-6d74f51c9c50 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:02.229224 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a26c78-bdff-4260-915c-6d74f51c9c50 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:02.230855 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 198/379] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 14:00:02.237285 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30db52a-1bfe-4087-9624-d0045f485811 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:02.237933 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30db52a-1bfe-4087-9624-d0045f485811 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:02.238660 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30db52a-1bfe-4087-9624-d0045f485811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:02.242744 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:02.274136 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-b30db52a-1bfe-4087-9624-d0045f485811 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:02.277078 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b30db52a-1bfe-4087-9624-d0045f485811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:02.314283 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944ce592-b0d6-47fa-98e4-41cea1aef1e5 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:02.319938 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:02.370567 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501112e-9abd-4c74-81be-9e3db801305c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:02.371398 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501112e-9abd-4c74-81be-9e3db801305c None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:02.371568 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501112e-9abd-4c74-81be-9e3db801305c None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:02.371810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501112e-9abd-4c74-81be-9e3db801305c None admin] PATH_INFO: `/v3/groups/e06ac03aadb7402884a9f12c228d338a` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:02.374329 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 199/380] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:02.388663 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501112e-9abd-4c74-81be-9e3db801305c None admin] RBAC: Authorizing `identity:get_group(group_id=e06ac03aadb7402884a9f12c228d338a)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:02.390489 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501112e-9abd-4c74-81be-9e3db801305c None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:02.392579 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 182/381] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Sep 12 14:00:02 2022] GET /identity/v3/groups/e06ac03aadb7402884a9f12c228d338a => generated 229 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:02.398812 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30db52a-1bfe-4087-9624-d0045f485811 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:02.533950 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30db52a-1bfe-4087-9624-d0045f485811 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:02.572188 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d6edd-185f-4092-8ca4-adefa508a146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:02.572799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d6edd-185f-4092-8ca4-adefa508a146 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:02.573076 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d6edd-185f-4092-8ca4-adefa508a146 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:02.573350 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d6edd-185f-4092-8ca4-adefa508a146 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:02.574140 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d6edd-185f-4092-8ca4-adefa508a146 None admin] RBAC: Adding query filter params (group.id=e06ac03aadb7402884a9f12c228d338a, role.id=e14564f00daf48a28d4e32da8cd2efad, scope.project.id=b159fcc4f05d45de88c679e235640813) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:02.574452 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d6edd-185f-4092-8ca4-adefa508a146 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:02.575947 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d6edd-185f-4092-8ca4-adefa508a146 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:02.585492 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 200/382] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Sep 12 14:00:02 2022] GET /identity/v3/role_assignments?group.id=e06ac03aadb7402884a9f12c228d338a&scope.project.id=b159fcc4f05d45de88c679e235640813&role.id=e14564f00daf48a28d4e32da8cd2efad => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:02.720918 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501112e-9abd-4c74-81be-9e3db801305c None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:02.835588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501112e-9abd-4c74-81be-9e3db801305c None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:02.848283 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68772741-916d-460a-9f24-85200a66d912 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:02.848616 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68772741-916d-460a-9f24-85200a66d912 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:02.848902 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68772741-916d-460a-9f24-85200a66d912 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:02.851510 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 201/383] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:02.853958 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:02.856999 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d39109-e31c-4c98-a12c-d7d9d043ec31 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:02.857283 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d39109-e31c-4c98-a12c-d7d9d043ec31 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:02.857581 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d39109-e31c-4c98-a12c-d7d9d043ec31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:02.877568 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-54d39109-e31c-4c98-a12c-d7d9d043ec31 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:02.878802 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-54d39109-e31c-4c98-a12c-d7d9d043ec31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:02.883354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412f6cf-d7eb-4dd0-b7d4-9c0fa7f23be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:02.884823 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412f6cf-d7eb-4dd0-b7d4-9c0fa7f23be0 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:02.885236 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412f6cf-d7eb-4dd0-b7d4-9c0fa7f23be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:02.885638 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412f6cf-d7eb-4dd0-b7d4-9c0fa7f23be0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:02.897802 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9412f6cf-d7eb-4dd0-b7d4-9c0fa7f23be0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:02.904048 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-9412f6cf-d7eb-4dd0-b7d4-9c0fa7f23be0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 14:00:02.904048 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:02.904048 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:02.904048 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:02.904048 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:02.904048 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:02.904048 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:02.904048 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:02.904048 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:02.904048 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:02.904048 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:02.904048 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:02.904048 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:02.904048 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:02.904048 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:02.904048 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:02.905149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:02.905149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:02.905149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:02.905149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:02.905149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:02.905149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:02.905149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:02.905149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:02.905149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:02.905149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:02.905149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:02.905149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:02.905149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:02.905149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:02.905149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:02.905149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:02.906135 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:02.906135 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:02.906135 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:02.906135 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:02.906135 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:02.906135 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:02.906135 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:02.906135 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:02.906135 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:02.906135 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:02.906135 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 14:00:02.906135 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:02.907225 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:02.909281 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 183/384] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 12 14:00:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:02.914897 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412f6cf-d7eb-4dd0-b7d4-9c0fa7f23be0 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:02.967220 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 202/385] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:03.054962 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412f6cf-d7eb-4dd0-b7d4-9c0fa7f23be0 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:03.095175 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f95c7b-a2ca-4589-818b-3e87a5c73abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:03.095790 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f95c7b-a2ca-4589-818b-3e87a5c73abf None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:03.096070 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f95c7b-a2ca-4589-818b-3e87a5c73abf None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:03.096339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f95c7b-a2ca-4589-818b-3e87a5c73abf None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:03.097006 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f95c7b-a2ca-4589-818b-3e87a5c73abf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:03.097304 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f95c7b-a2ca-4589-818b-3e87a5c73abf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:03.098678 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f95c7b-a2ca-4589-818b-3e87a5c73abf None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:03.110284 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 184/386] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:00:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:03.115550 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d39109-e31c-4c98-a12c-d7d9d043ec31 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:03.253802 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d39109-e31c-4c98-a12c-d7d9d043ec31 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:03.297006 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dbbacc-d1dd-40d8-8465-04586a31552d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:03.297637 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbbacc-d1dd-40d8-8465-04586a31552d None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:03.297911 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbbacc-d1dd-40d8-8465-04586a31552d None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:03.298178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbbacc-d1dd-40d8-8465-04586a31552d None admin] PATH_INFO: `/v3/groups/2378a09b5ad64b1eb9d19d3c3d05296a` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:03.303514 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dbbacc-d1dd-40d8-8465-04586a31552d None admin] RBAC: Authorizing `identity:get_group(group_id=2378a09b5ad64b1eb9d19d3c3d05296a)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:03.304958 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dbbacc-d1dd-40d8-8465-04586a31552d None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:03.307491 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 203/387] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Sep 12 14:00:03 2022] GET /identity/v3/groups/2378a09b5ad64b1eb9d19d3c3d05296a => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:03.312577 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f95c7b-a2ca-4589-818b-3e87a5c73abf None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:03.363822 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:03.405461 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dbbacc-d1dd-40d8-8465-04586a31552d None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:03.448231 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:03.454804 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f95c7b-a2ca-4589-818b-3e87a5c73abf None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:03.516050 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70714bd5-64a2-432e-95f8-8866652dd2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:03.516693 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70714bd5-64a2-432e-95f8-8866652dd2be None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:03.516964 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70714bd5-64a2-432e-95f8-8866652dd2be None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:03.517231 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70714bd5-64a2-432e-95f8-8866652dd2be None admin] PATH_INFO: `/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:03.518535 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70714bd5-64a2-432e-95f8-8866652dd2be None admin] RBAC: Authorizing `identity:get_project(project_id=e10c65c7a17c44a7a8711e7701efdcd0)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:03.521338 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70714bd5-64a2-432e-95f8-8866652dd2be None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:03.523006 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 185/388] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 14:00:03 2022] GET /identity/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:03.530573 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70714bd5-64a2-432e-95f8-8866652dd2be None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:03.560005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dbbacc-d1dd-40d8-8465-04586a31552d None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:03.624710 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96e39a4-dd80-42ea-9576-97c6f209f107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:03.625427 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e39a4-dd80-42ea-9576-97c6f209f107 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:03.625706 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e39a4-dd80-42ea-9576-97c6f209f107 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:03.625976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e39a4-dd80-42ea-9576-97c6f209f107 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:03.645605 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96e39a4-dd80-42ea-9576-97c6f209f107 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:03.655999 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-b96e39a4-dd80-42ea-9576-97c6f209f107 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 14:00:03.655999 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:03.655999 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:03.655999 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:03.655999 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:03.655999 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:03.655999 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:03.655999 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:03.655999 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:03.655999 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:03.655999 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:03.655999 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:03.655999 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:03.655999 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:03.655999 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:03.655999 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:03.656961 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:03.656961 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:03.656961 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:03.656961 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:03.656961 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:03.656961 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:03.656961 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:03.656961 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:03.656961 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:03.656961 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:03.656961 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:03.656961 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:03.656961 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:03.656961 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:03.656961 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:03.656961 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:03.657756 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:03.657756 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:03.657756 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:03.657756 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:03.657756 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:03.657756 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:03.657756 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:03.657756 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:03.657756 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:03.657756 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:03.657756 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 14:00:03.657756 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:03.657756 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 204/389] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 14:00:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:00:03.666418 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96e39a4-dd80-42ea-9576-97c6f209f107 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:03.711173 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:03.716356 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70714bd5-64a2-432e-95f8-8866652dd2be None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:03.779624 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb69ea-88ed-48b4-8a38-6e3bfc6c8754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:03.780301 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb69ea-88ed-48b4-8a38-6e3bfc6c8754 None admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:03.780581 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb69ea-88ed-48b4-8a38-6e3bfc6c8754 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:03.780853 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb69ea-88ed-48b4-8a38-6e3bfc6c8754 None admin] PATH_INFO: `/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0/groups/2378a09b5ad64b1eb9d19d3c3d05296a/roles/bcde1b76af8048deb2829c5aaa27296c` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:03.785109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb69ea-88ed-48b4-8a38-6e3bfc6c8754 None admin] RBAC: Authorizing `identity:create_grant(project_id=e10c65c7a17c44a7a8711e7701efdcd0, group_id=2378a09b5ad64b1eb9d19d3c3d05296a, role_id=bcde1b76af8048deb2829c5aaa27296c)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:03.786548 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb69ea-88ed-48b4-8a38-6e3bfc6c8754 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:03.826631 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:03.841050 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 186/390] 10.222.0.42 () {58 vars in 1442 bytes} [Mon Sep 12 14:00:03 2022] PUT /identity/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0/groups/2378a09b5ad64b1eb9d19d3c3d05296a/roles/bcde1b76af8048deb2829c5aaa27296c => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:00:03.850631 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96e39a4-dd80-42ea-9576-97c6f209f107 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:03.899652 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0c02a-3485-4d88-9cf4-12a77d0db4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:03.900876 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0c02a-3485-4d88-9cf4-12a77d0db4d0 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:03.901151 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0c02a-3485-4d88-9cf4-12a77d0db4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:03.901420 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0c02a-3485-4d88-9cf4-12a77d0db4d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:03.902097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0c02a-3485-4d88-9cf4-12a77d0db4d0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:03.902391 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0c02a-3485-4d88-9cf4-12a77d0db4d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:03.903804 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0c02a-3485-4d88-9cf4-12a77d0db4d0 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:03.913126 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 205/391] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 12 14:00:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:03.919070 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb69ea-88ed-48b4-8a38-6e3bfc6c8754 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:04.051142 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb69ea-88ed-48b4-8a38-6e3bfc6c8754 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:04.105925 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669c7e9-19e4-462d-875d-f59874bacbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:04.107526 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669c7e9-19e4-462d-875d-f59874bacbc4 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:04.107526 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669c7e9-19e4-462d-875d-f59874bacbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:04.107526 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669c7e9-19e4-462d-875d-f59874bacbc4 None admin] PATH_INFO: `/v3/groups/2378a09b5ad64b1eb9d19d3c3d05296a` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:04.108167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9669c7e9-19e4-462d-875d-f59874bacbc4 None admin] RBAC: Authorizing `identity:get_group(group_id=2378a09b5ad64b1eb9d19d3c3d05296a)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:04.109589 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9669c7e9-19e4-462d-875d-f59874bacbc4 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:04.111530 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 187/392] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Sep 12 14:00:03 2022] GET /identity/v3/groups/2378a09b5ad64b1eb9d19d3c3d05296a => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:04.116716 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0c02a-3485-4d88-9cf4-12a77d0db4d0 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:04.230656 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0c02a-3485-4d88-9cf4-12a77d0db4d0 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:04.272463 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45590d-ff60-4d18-8d3c-71937393fab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:04.273104 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45590d-ff60-4d18-8d3c-71937393fab0 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:04.273383 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45590d-ff60-4d18-8d3c-71937393fab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:04.273655 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45590d-ff60-4d18-8d3c-71937393fab0 None admin] PATH_INFO: `/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:04.274918 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45590d-ff60-4d18-8d3c-71937393fab0 None admin] RBAC: Authorizing `identity:get_project(project_id=e10c65c7a17c44a7a8711e7701efdcd0)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:04.276709 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45590d-ff60-4d18-8d3c-71937393fab0 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:04.278551 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 206/393] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 14:00:04 2022] GET /identity/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:04.284025 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669c7e9-19e4-462d-875d-f59874bacbc4 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:04.316571 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:04.417618 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669c7e9-19e4-462d-875d-f59874bacbc4 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:04.458697 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3902f58-54f9-4aa2-bf20-6fdf8aa0fc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:04.459445 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3902f58-54f9-4aa2-bf20-6fdf8aa0fc1f None admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:04.459634 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3902f58-54f9-4aa2-bf20-6fdf8aa0fc1f None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:04.459902 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3902f58-54f9-4aa2-bf20-6fdf8aa0fc1f None admin] PATH_INFO: `/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0/groups/2378a09b5ad64b1eb9d19d3c3d05296a/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:04.462926 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3902f58-54f9-4aa2-bf20-6fdf8aa0fc1f None admin] RBAC: Authorizing `identity:create_grant(project_id=e10c65c7a17c44a7a8711e7701efdcd0, group_id=2378a09b5ad64b1eb9d19d3c3d05296a, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:04.464369 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3902f58-54f9-4aa2-bf20-6fdf8aa0fc1f None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:04.494337 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 188/394] 10.222.0.42 () {58 vars in 1442 bytes} [Mon Sep 12 14:00:04 2022] PUT /identity/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0/groups/2378a09b5ad64b1eb9d19d3c3d05296a/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:00:04.511235 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d84e33-a00b-47cb-abbc-d062eb68c259 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:04.511535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d84e33-a00b-47cb-abbc-d062eb68c259 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:04.511834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d84e33-a00b-47cb-abbc-d062eb68c259 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:04.513176 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 207/395] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:04.518763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad72542-b4f9-4000-94aa-309721215647 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:04.519070 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad72542-b4f9-4000-94aa-309721215647 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:04.519347 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad72542-b4f9-4000-94aa-309721215647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:04.545848 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-0ad72542-b4f9-4000-94aa-309721215647 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:04.547143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-0ad72542-b4f9-4000-94aa-309721215647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:04.635359 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 189/396] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:05.001801 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d84e33-a00b-47cb-abbc-d062eb68c259 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:05.025520 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:05.126627 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d84e33-a00b-47cb-abbc-d062eb68c259 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:05.165191 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54527858-41d5-44fd-9c37-90e58ad9a14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:05.166381 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54527858-41d5-44fd-9c37-90e58ad9a14a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:05.166652 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54527858-41d5-44fd-9c37-90e58ad9a14a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:05.166938 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54527858-41d5-44fd-9c37-90e58ad9a14a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:05.183191 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54527858-41d5-44fd-9c37-90e58ad9a14a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:05.188633 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-54527858-41d5-44fd-9c37-90e58ad9a14a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 14:00:05.188633 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:05.188633 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:05.188633 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:05.188633 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:05.188633 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:05.188633 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:05.188633 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:05.188633 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:05.188633 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:05.188633 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:05.188633 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:05.188633 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:05.188633 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:05.188633 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:05.188633 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:05.189505 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:05.189505 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:05.189505 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:05.189505 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:05.189505 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:05.189505 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:05.189505 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:05.189505 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:05.189505 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:05.189505 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:05.189505 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:05.189505 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:05.189505 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:05.189505 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:05.189505 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:05.189505 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:05.190294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:05.190294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:05.190294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:05.190294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:05.190294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:05.190294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:05.190294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:05.190294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:05.190294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:05.190294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:05.190294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 14:00:05.190294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:05.190294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 208/397] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 14:00:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:05.197518 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad72542-b4f9-4000-94aa-309721215647 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:05.234499 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:05.355154 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad72542-b4f9-4000-94aa-309721215647 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:05.366170 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde6034-7f7c-49c4-afed-ac203c5bb187 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:05.366488 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde6034-7f7c-49c4-afed-ac203c5bb187 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:05.366721 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde6034-7f7c-49c4-afed-ac203c5bb187 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:05.368358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 209/398] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:05.380522 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5b921-72f5-42c4-a753-43c73abd0ece None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:05.380710 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5b921-72f5-42c4-a753-43c73abd0ece None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:05.380966 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5b921-72f5-42c4-a753-43c73abd0ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:05.402616 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b0ba1-40b0-4c39-bb2d-d8be253a5037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:05.403591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b0ba1-40b0-4c39-bb2d-d8be253a5037 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:05.404092 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b0ba1-40b0-4c39-bb2d-d8be253a5037 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:05.404562 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b0ba1-40b0-4c39-bb2d-d8be253a5037 None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:05.405425 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b0ba1-40b0-4c39-bb2d-d8be253a5037 None admin] RBAC: Adding query filter params (name=member) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:05.410522 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b0ba1-40b0-4c39-bb2d-d8be253a5037 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:05.412163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b0ba1-40b0-4c39-bb2d-d8be253a5037 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:05.422351 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 190/399] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 12 14:00:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:05.423643 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-50d5b921-72f5-42c4-a753-43c73abd0ece None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:05.425043 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-50d5b921-72f5-42c4-a753-43c73abd0ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:05.427408 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b0ba1-40b0-4c39-bb2d-d8be253a5037 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:05.549469 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:05.550226 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 210/400] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 14:00:05.551746 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ca73b-1097-48e7-917c-5ebccfd0ac73 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:05.552224 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ca73b-1097-48e7-917c-5ebccfd0ac73 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:05.552672 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ca73b-1097-48e7-917c-5ebccfd0ac73 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:05.554301 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 211/401] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:05.562497 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46983449-4166-467b-ba4f-1450ee060f0f None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:05.562775 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46983449-4166-467b-ba4f-1450ee060f0f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:05.563079 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46983449-4166-467b-ba4f-1450ee060f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:05.576768 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b0ba1-40b0-4c39-bb2d-d8be253a5037 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:05.584960 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-46983449-4166-467b-ba4f-1450ee060f0f None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:05.586174 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-46983449-4166-467b-ba4f-1450ee060f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:05.589317 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:05.625034 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976c5b9-0360-4b9c-b712-c7b83a18ac01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:05.626053 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976c5b9-0360-4b9c-b712-c7b83a18ac01 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:05.626545 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976c5b9-0360-4b9c-b712-c7b83a18ac01 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:05.627791 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976c5b9-0360-4b9c-b712-c7b83a18ac01 None admin] PATH_INFO: `/v3/groups/2378a09b5ad64b1eb9d19d3c3d05296a` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:05.629985 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5976c5b9-0360-4b9c-b712-c7b83a18ac01 None admin] RBAC: Authorizing `identity:get_group(group_id=2378a09b5ad64b1eb9d19d3c3d05296a)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:05.633595 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5976c5b9-0360-4b9c-b712-c7b83a18ac01 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:05.635482 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 191/402] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Sep 12 14:00:05 2022] GET /identity/v3/groups/2378a09b5ad64b1eb9d19d3c3d05296a => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:00:05.641104 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976c5b9-0360-4b9c-b712-c7b83a18ac01 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:05.710718 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 212/403] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:05.787244 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976c5b9-0360-4b9c-b712-c7b83a18ac01 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:05.837223 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1e969-8bee-4b54-b1af-c4f0bd193e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:05.837223 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1e969-8bee-4b54-b1af-c4f0bd193e1b None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:05.837223 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1e969-8bee-4b54-b1af-c4f0bd193e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:05.837223 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1e969-8bee-4b54-b1af-c4f0bd193e1b None admin] PATH_INFO: `/v3/projects/6606f4e0816344eca4228a4e9d0ed52e` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:05.843232 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea1e969-8bee-4b54-b1af-c4f0bd193e1b None admin] RBAC: Authorizing `identity:get_project(project_id=6606f4e0816344eca4228a4e9d0ed52e)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:05.844731 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea1e969-8bee-4b54-b1af-c4f0bd193e1b None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:05.847133 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 192/404] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 14:00:05 2022] GET /identity/v3/projects/6606f4e0816344eca4228a4e9d0ed52e => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:05.852873 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46983449-4166-467b-ba4f-1450ee060f0f None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:05.933051 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1e969-8bee-4b54-b1af-c4f0bd193e1b None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:05.963094 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:05.982928 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46983449-4166-467b-ba4f-1450ee060f0f None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:06.016520 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ac908-e407-48b2-bea5-cd523330d10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:06.017213 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ac908-e407-48b2-bea5-cd523330d10d None admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:06.017442 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ac908-e407-48b2-bea5-cd523330d10d None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:06.017707 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ac908-e407-48b2-bea5-cd523330d10d None admin] PATH_INFO: `/v3/projects/6606f4e0816344eca4228a4e9d0ed52e/groups/2378a09b5ad64b1eb9d19d3c3d05296a/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:06.019603 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0ac908-e407-48b2-bea5-cd523330d10d None admin] RBAC: Authorizing `identity:create_grant(project_id=6606f4e0816344eca4228a4e9d0ed52e, group_id=2378a09b5ad64b1eb9d19d3c3d05296a, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:06.020987 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0ac908-e407-48b2-bea5-cd523330d10d None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:06.030233 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:06.040089 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 213/405] 10.222.0.42 () {58 vars in 1442 bytes} [Mon Sep 12 14:00:05 2022] PUT /identity/v3/projects/6606f4e0816344eca4228a4e9d0ed52e/groups/2378a09b5ad64b1eb9d19d3c3d05296a/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:00:06.086014 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1e969-8bee-4b54-b1af-c4f0bd193e1b None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:06.116946 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ac908-e407-48b2-bea5-cd523330d10d None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:06.124842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9225ce6-0e36-449b-9a8a-9c479bc2abfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:06.125821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9225ce6-0e36-449b-9a8a-9c479bc2abfc None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:06.126295 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9225ce6-0e36-449b-9a8a-9c479bc2abfc None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:06.126884 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9225ce6-0e36-449b-9a8a-9c479bc2abfc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:06.143385 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9225ce6-0e36-449b-9a8a-9c479bc2abfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:06.149568 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-f9225ce6-0e36-449b-9a8a-9c479bc2abfc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:00:06.149568 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:06.149568 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:06.149568 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:06.149568 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:06.149568 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:06.149568 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:06.149568 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:06.149568 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:06.149568 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:06.149568 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:06.149568 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:06.149568 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:06.149568 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:06.149568 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:06.149568 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:06.150543 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:06.150543 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:06.150543 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:06.150543 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:06.150543 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:06.150543 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:06.150543 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:06.150543 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:06.150543 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:06.150543 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:06.150543 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:06.150543 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:06.150543 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:06.150543 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:06.150543 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:06.150543 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:06.151354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:06.151354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:06.151354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:06.151354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:06.151354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:06.151354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:06.151354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:06.151354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:06.151354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:06.151354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:06.151354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:00:06.151354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:06.154206 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 193/406] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 12 14:00:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:06.164072 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9225ce6-0e36-449b-9a8a-9c479bc2abfc None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:06.273810 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ac908-e407-48b2-bea5-cd523330d10d None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:06.305848 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:06.316623 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada29fb9-8b99-46dd-8eeb-7a3e8c406485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:06.317564 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada29fb9-8b99-46dd-8eeb-7a3e8c406485 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:06.318044 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada29fb9-8b99-46dd-8eeb-7a3e8c406485 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:06.318508 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada29fb9-8b99-46dd-8eeb-7a3e8c406485 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:06.331522 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada29fb9-8b99-46dd-8eeb-7a3e8c406485 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:06.333596 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9225ce6-0e36-449b-9a8a-9c479bc2abfc None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:06.337104 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-ada29fb9-8b99-46dd-8eeb-7a3e8c406485 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 14:00:06.337104 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:06.337104 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:06.337104 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:06.337104 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:06.337104 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:06.337104 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:06.337104 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:06.337104 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:06.337104 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:06.337104 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:06.337104 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:06.337104 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:06.337104 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:06.337104 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:06.337104 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:06.338091 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:06.338091 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:06.338091 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:06.338091 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:06.338091 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:06.338091 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:06.338091 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:06.338091 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:06.338091 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:06.338091 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:06.338091 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:06.338091 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:06.338091 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:06.338091 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:06.338091 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:06.338091 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:06.338909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:06.338909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:06.338909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:06.338909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:06.338909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:06.338909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:06.338909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:06.338909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:06.338909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:06.338909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:06.338909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 14:00:06.338909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:06.341558 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 214/407] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 12 14:00:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:00:06.350067 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada29fb9-8b99-46dd-8eeb-7a3e8c406485 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:06.395847 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980b482-77ed-4172-8e67-3e55a0d9c291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:06.396459 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980b482-77ed-4172-8e67-3e55a0d9c291 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:06.396740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980b482-77ed-4172-8e67-3e55a0d9c291 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:06.397003 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980b482-77ed-4172-8e67-3e55a0d9c291 None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:06.397666 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1980b482-77ed-4172-8e67-3e55a0d9c291 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:06.397961 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1980b482-77ed-4172-8e67-3e55a0d9c291 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:06.399358 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1980b482-77ed-4172-8e67-3e55a0d9c291 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:06.408715 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 194/408] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 14:00:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:06.413702 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980b482-77ed-4172-8e67-3e55a0d9c291 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:06.469243 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:06.562952 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada29fb9-8b99-46dd-8eeb-7a3e8c406485 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:06.594182 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980b482-77ed-4172-8e67-3e55a0d9c291 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:06.629199 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd971231-5fd2-4259-90e5-2244fa0e6067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:06.629835 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd971231-5fd2-4259-90e5-2244fa0e6067 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:06.630112 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd971231-5fd2-4259-90e5-2244fa0e6067 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:06.630382 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd971231-5fd2-4259-90e5-2244fa0e6067 None admin] PATH_INFO: `/v3/groups/e06ac03aadb7402884a9f12c228d338a` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:06.631765 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd971231-5fd2-4259-90e5-2244fa0e6067 None admin] RBAC: Authorizing `identity:get_group(group_id=e06ac03aadb7402884a9f12c228d338a)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:06.633211 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd971231-5fd2-4259-90e5-2244fa0e6067 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:06.634863 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 195/409] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Sep 12 14:00:06 2022] GET /identity/v3/groups/e06ac03aadb7402884a9f12c228d338a => generated 229 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:00:06.640938 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79200388-6dd1-4179-aad0-ae9daa299dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:06.641128 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd971231-5fd2-4259-90e5-2244fa0e6067 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:06.641504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79200388-6dd1-4179-aad0-ae9daa299dfc None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:06.641780 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79200388-6dd1-4179-aad0-ae9daa299dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:06.642051 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79200388-6dd1-4179-aad0-ae9daa299dfc None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:06.642731 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79200388-6dd1-4179-aad0-ae9daa299dfc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:06.643061 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79200388-6dd1-4179-aad0-ae9daa299dfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:06.644431 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79200388-6dd1-4179-aad0-ae9daa299dfc None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:06.651193 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:06.653369 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 215/410] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:00:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:06.659023 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79200388-6dd1-4179-aad0-ae9daa299dfc None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:06.757924 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd971231-5fd2-4259-90e5-2244fa0e6067 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:06.825776 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9d996-e8b4-4f03-8c02-965fe83c9352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:06.826350 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9d996-e8b4-4f03-8c02-965fe83c9352 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:06.826625 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9d996-e8b4-4f03-8c02-965fe83c9352 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:06.826914 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9d996-e8b4-4f03-8c02-965fe83c9352 None admin] PATH_INFO: `/v3/projects/b159fcc4f05d45de88c679e235640813` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:06.828049 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f9d996-e8b4-4f03-8c02-965fe83c9352 None admin] RBAC: Authorizing `identity:get_project(project_id=b159fcc4f05d45de88c679e235640813)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:06.830517 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f9d996-e8b4-4f03-8c02-965fe83c9352 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:06.832505 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 196/411] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 14:00:06 2022] GET /identity/v3/projects/b159fcc4f05d45de88c679e235640813 => generated 344 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:06.841132 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9d996-e8b4-4f03-8c02-965fe83c9352 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:06.868227 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79200388-6dd1-4179-aad0-ae9daa299dfc None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:06.892278 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:06.907783 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e9850-7520-4e03-811e-eec6627290ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:06.908415 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e9850-7520-4e03-811e-eec6627290ef None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:06.908691 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e9850-7520-4e03-811e-eec6627290ef None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:06.908961 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e9850-7520-4e03-811e-eec6627290ef None admin] PATH_INFO: `/v3/groups/2378a09b5ad64b1eb9d19d3c3d05296a` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:06.910062 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168e9850-7520-4e03-811e-eec6627290ef None admin] RBAC: Authorizing `identity:get_group(group_id=2378a09b5ad64b1eb9d19d3c3d05296a)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:06.911596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168e9850-7520-4e03-811e-eec6627290ef None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:06.913497 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 216/412] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Sep 12 14:00:06 2022] GET /identity/v3/groups/2378a09b5ad64b1eb9d19d3c3d05296a => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:06.914691 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7268afe-943e-4c10-9d5b-ea0268831a64 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:06.919996 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7268afe-943e-4c10-9d5b-ea0268831a64 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:06.920304 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7268afe-943e-4c10-9d5b-ea0268831a64 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:06.921882 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 217/413] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:06.926200 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7268afe-943e-4c10-9d5b-ea0268831a64 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:06.965294 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:07.002893 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9d996-e8b4-4f03-8c02-965fe83c9352 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:07.033443 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7268afe-943e-4c10-9d5b-ea0268831a64 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:07.054701 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ee9d4-7edb-44df-b5fc-eb2d091480ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:07.055702 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ee9d4-7edb-44df-b5fc-eb2d091480ad None admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:07.056728 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ee9d4-7edb-44df-b5fc-eb2d091480ad None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:07.057212 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ee9d4-7edb-44df-b5fc-eb2d091480ad None admin] PATH_INFO: `/v3/projects/b159fcc4f05d45de88c679e235640813/groups/e06ac03aadb7402884a9f12c228d338a/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:07.060359 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ee9d4-7edb-44df-b5fc-eb2d091480ad None admin] RBAC: Authorizing `identity:create_grant(project_id=b159fcc4f05d45de88c679e235640813, group_id=e06ac03aadb7402884a9f12c228d338a, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:07.062542 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ee9d4-7edb-44df-b5fc-eb2d091480ad None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:07.071297 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b924c9-b6d1-454c-964c-dedcadfa717d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:07.072366 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b924c9-b6d1-454c-964c-dedcadfa717d None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:07.072643 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b924c9-b6d1-454c-964c-dedcadfa717d None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:07.072920 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b924c9-b6d1-454c-964c-dedcadfa717d None admin] PATH_INFO: `/v3/projects/6606f4e0816344eca4228a4e9d0ed52e` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:07.074000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b924c9-b6d1-454c-964c-dedcadfa717d None admin] RBAC: Authorizing `identity:get_project(project_id=6606f4e0816344eca4228a4e9d0ed52e)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:07.075686 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b924c9-b6d1-454c-964c-dedcadfa717d None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:07.077489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 218/414] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 14:00:06 2022] GET /identity/v3/projects/6606f4e0816344eca4228a4e9d0ed52e => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:07.078579 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fda571-27f7-4822-9086-df8c8967de98 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:07.078787 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fda571-27f7-4822-9086-df8c8967de98 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:07.082321 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 197/415] 10.222.0.42 () {58 vars in 1442 bytes} [Mon Sep 12 14:00:06 2022] PUT /identity/v3/projects/b159fcc4f05d45de88c679e235640813/groups/e06ac03aadb7402884a9f12c228d338a/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:00:07.087885 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fda571-27f7-4822-9086-df8c8967de98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:07.090352 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ee9d4-7edb-44df-b5fc-eb2d091480ad None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:07.111405 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-84fda571-27f7-4822-9086-df8c8967de98 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:07.113761 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-84fda571-27f7-4822-9086-df8c8967de98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:07.191723 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:07.220408 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 219/416] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:07.294547 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f0d71-61de-4377-9711-3f3b34d7c062 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:07.294880 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f0d71-61de-4377-9711-3f3b34d7c062 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:07.295141 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f0d71-61de-4377-9711-3f3b34d7c062 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:07.296778 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 220/417] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 14:00:07.301784 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9188a-f8bb-4d86-8032-1ee6f3be3aa1 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:07.302092 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9188a-f8bb-4d86-8032-1ee6f3be3aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:07.302340 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9188a-f8bb-4d86-8032-1ee6f3be3aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:07.309599 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ee9d4-7edb-44df-b5fc-eb2d091480ad None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:07.322830 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-71e9188a-f8bb-4d86-8032-1ee6f3be3aa1 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:07.324062 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-71e9188a-f8bb-4d86-8032-1ee6f3be3aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:07.384577 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b3e08-1843-439a-9e5b-905c29c79112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:07.385232 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b3e08-1843-439a-9e5b-905c29c79112 None admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:07.385502 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b3e08-1843-439a-9e5b-905c29c79112 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:07.385766 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b3e08-1843-439a-9e5b-905c29c79112 None admin] PATH_INFO: `/v3/projects/6606f4e0816344eca4228a4e9d0ed52e/groups/2378a09b5ad64b1eb9d19d3c3d05296a/roles/bcde1b76af8048deb2829c5aaa27296c` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:07.392736 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563b3e08-1843-439a-9e5b-905c29c79112 None admin] RBAC: Authorizing `identity:create_grant(project_id=6606f4e0816344eca4228a4e9d0ed52e, group_id=2378a09b5ad64b1eb9d19d3c3d05296a, role_id=bcde1b76af8048deb2829c5aaa27296c)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:07.394181 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563b3e08-1843-439a-9e5b-905c29c79112 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:07.399141 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:07.418891 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 221/419] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:07.419460 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 198/419] 10.222.0.42 () {58 vars in 1442 bytes} [Mon Sep 12 14:00:07 2022] PUT /identity/v3/projects/6606f4e0816344eca4228a4e9d0ed52e/groups/2378a09b5ad64b1eb9d19d3c3d05296a/roles/bcde1b76af8048deb2829c5aaa27296c => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:00:07.531078 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e9188a-f8bb-4d86-8032-1ee6f3be3aa1 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:07.650514 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e9188a-f8bb-4d86-8032-1ee6f3be3aa1 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:07.669903 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:07.688998 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea4f0f-a88d-46cd-ab23-9954813064c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:07.689636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea4f0f-a88d-46cd-ab23-9954813064c1 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:07.689909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea4f0f-a88d-46cd-ab23-9954813064c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:07.690175 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea4f0f-a88d-46cd-ab23-9954813064c1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:07.702427 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea4f0f-a88d-46cd-ab23-9954813064c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:07.707396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-b2ea4f0f-a88d-46cd-ab23-9954813064c1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 14:00:07.707396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:07.707396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:07.707396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:07.707396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:07.707396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:07.707396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:07.707396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:07.707396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:07.707396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:07.707396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:07.707396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:07.707396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:07.707396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:07.707396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:07.707396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:07.708273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:07.708273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:07.708273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:07.708273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:07.708273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:07.708273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:07.708273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:07.708273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:07.708273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:07.708273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:07.708273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:07.708273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:07.708273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:07.708273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:07.708273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:07.708273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:07.709064 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:07.709064 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:07.709064 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:07.709064 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:07.709064 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:07.709064 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:07.709064 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:07.709064 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:07.709064 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:07.709064 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:07.709064 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 14:00:07.709064 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:07.709064 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 222/420] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 12 14:00:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:07.717908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b3e08-1843-439a-9e5b-905c29c79112 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:07.740744 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea4f0f-a88d-46cd-ab23-9954813064c1 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:07.876208 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b3e08-1843-439a-9e5b-905c29c79112 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:07.890737 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea4f0f-a88d-46cd-ab23-9954813064c1 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:07.945676 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d61285-cd59-47f0-841c-ff86d5b7bc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:07.946200 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d61285-cd59-47f0-841c-ff86d5b7bc2f None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:07.947059 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d61285-cd59-47f0-841c-ff86d5b7bc2f None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:07.947357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d61285-cd59-47f0-841c-ff86d5b7bc2f None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:07.948024 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d61285-cd59-47f0-841c-ff86d5b7bc2f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:07.948318 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d61285-cd59-47f0-841c-ff86d5b7bc2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:07.949683 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d61285-cd59-47f0-841c-ff86d5b7bc2f None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:07.954648 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8663f9b-4d78-4c92-b04d-3b6cceaea0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:07.955309 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8663f9b-4d78-4c92-b04d-3b6cceaea0d0 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:07.955582 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8663f9b-4d78-4c92-b04d-3b6cceaea0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:07.955854 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8663f9b-4d78-4c92-b04d-3b6cceaea0d0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:07.968239 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 199/421] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:00:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:07.977067 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d61285-cd59-47f0-841c-ff86d5b7bc2f None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:07.977998 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8663f9b-4d78-4c92-b04d-3b6cceaea0d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:07.983147 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-a8663f9b-4d78-4c92-b04d-3b6cceaea0d0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 14:00:07.983147 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:07.983147 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:07.983147 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:07.983147 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:07.983147 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:07.983147 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:07.983147 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:07.983147 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:07.983147 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:07.983147 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:07.983147 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:07.983147 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:07.983147 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:07.983147 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:07.983147 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:07.984014 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:07.984014 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:07.984014 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:07.984014 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:07.984014 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:07.984014 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:07.984014 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:07.984014 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:07.984014 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:07.984014 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:07.984014 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:07.984014 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:07.984014 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:07.984014 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:07.984014 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:07.984014 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:07.984828 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:07.984828 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:07.984828 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:07.984828 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:07.984828 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:07.984828 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:07.984828 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:07.984828 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:07.984828 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:07.984828 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:07.984828 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 14:00:07.984828 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:07.984828 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 223/422] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 14:00:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:00:07.991304 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8663f9b-4d78-4c92-b04d-3b6cceaea0d0 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:08.029298 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:08.091620 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:08.138374 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8663f9b-4d78-4c92-b04d-3b6cceaea0d0 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:08.186131 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d61285-cd59-47f0-841c-ff86d5b7bc2f None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:08.211462 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ec868-073c-476a-aada-8255f86613e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:08.212087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ec868-073c-476a-aada-8255f86613e7 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:08.212536 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ec868-073c-476a-aada-8255f86613e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:08.212813 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ec868-073c-476a-aada-8255f86613e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:08.213484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915ec868-073c-476a-aada-8255f86613e7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:08.213788 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915ec868-073c-476a-aada-8255f86613e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:08.215301 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915ec868-073c-476a-aada-8255f86613e7 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:08.223432 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbeedac-ed2b-496e-8ed8-5f861a3aa64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:08.224543 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbeedac-ed2b-496e-8ed8-5f861a3aa64f None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:08.224817 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbeedac-ed2b-496e-8ed8-5f861a3aa64f None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:08.225081 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbeedac-ed2b-496e-8ed8-5f861a3aa64f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:08.237133 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 224/423] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 12 14:00:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:08.247024 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ec868-073c-476a-aada-8255f86613e7 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:08.314593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbeedac-ed2b-496e-8ed8-5f861a3aa64f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:08.316602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbeedac-ed2b-496e-8ed8-5f861a3aa64f None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:08.437934 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:08.456007 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:08.462007 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ec868-073c-476a-aada-8255f86613e7 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:08.535288 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 200/424] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:00:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 14:00:08.541193 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbeedac-ed2b-496e-8ed8-5f861a3aa64f None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:08.543736 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f67367-e5f0-4355-96f7-7501329777ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:08.544488 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f67367-e5f0-4355-96f7-7501329777ae None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:08.544773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f67367-e5f0-4355-96f7-7501329777ae None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:08.545049 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f67367-e5f0-4355-96f7-7501329777ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:08.654939 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f67367-e5f0-4355-96f7-7501329777ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:08.656326 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f67367-e5f0-4355-96f7-7501329777ae None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:08.766914 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbeedac-ed2b-496e-8ed8-5f861a3aa64f None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:08.809040 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 225/425] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:00:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 12 14:00:08.811758 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c79c194-6646-4103-8b78-6d6364bc0d54 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:08.812833 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c79c194-6646-4103-8b78-6d6364bc0d54 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:08.813889 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c79c194-6646-4103-8b78-6d6364bc0d54 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:08.824566 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745676f1-ac04-4ebe-b37c-7e4c456a3108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:08.825569 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745676f1-ac04-4ebe-b37c-7e4c456a3108 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:08.830919 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745676f1-ac04-4ebe-b37c-7e4c456a3108 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:08.830919 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745676f1-ac04-4ebe-b37c-7e4c456a3108 None admin] PATH_INFO: `/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:08.830919 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745676f1-ac04-4ebe-b37c-7e4c456a3108 None admin] RBAC: Authorizing `identity:get_project(project_id=e10c65c7a17c44a7a8711e7701efdcd0)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:08.831304 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745676f1-ac04-4ebe-b37c-7e4c456a3108 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:08.831913 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 226/426] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:08 2022] GET /identity => generated 270 bytes in 22 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:08.833540 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c79c194-6646-4103-8b78-6d6364bc0d54 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:08.834278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 201/427] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 14:00:08 2022] GET /identity/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0 => generated 298 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:08.841099 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:08.846171 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745676f1-ac04-4ebe-b37c-7e4c456a3108 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:08.877622 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:08.962988 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c79c194-6646-4103-8b78-6d6364bc0d54 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:08.972124 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745676f1-ac04-4ebe-b37c-7e4c456a3108 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:09.000901 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad08322-e054-4781-9906-3a1064d6c87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:09.001864 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad08322-e054-4781-9906-3a1064d6c87e None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:09.002448 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad08322-e054-4781-9906-3a1064d6c87e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:09.002964 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad08322-e054-4781-9906-3a1064d6c87e None admin] PATH_INFO: `/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:09.004357 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad08322-e054-4781-9906-3a1064d6c87e None admin] RBAC: Authorizing `identity:get_project(project_id=e10c65c7a17c44a7a8711e7701efdcd0)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:09.006204 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad08322-e054-4781-9906-3a1064d6c87e None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:09.008479 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 227/428] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 14:00:08 2022] GET /identity/v3/projects/e10c65c7a17c44a7a8711e7701efdcd0 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:09.010391 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663643fb-5d80-4918-90e9-2014bc17bbef None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:09.010741 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663643fb-5d80-4918-90e9-2014bc17bbef None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:09.011123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663643fb-5d80-4918-90e9-2014bc17bbef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:09.015453 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c22156-f954-4289-805f-acc059ddc7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:09.016101 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c22156-f954-4289-805f-acc059ddc7b6 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:09.016375 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c22156-f954-4289-805f-acc059ddc7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:09.016641 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c22156-f954-4289-805f-acc059ddc7b6 None admin] PATH_INFO: `/v3/groups/2378a09b5ad64b1eb9d19d3c3d05296a` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:09.017835 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c22156-f954-4289-805f-acc059ddc7b6 None admin] RBAC: Authorizing `identity:get_group(group_id=2378a09b5ad64b1eb9d19d3c3d05296a)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:09.020186 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c22156-f954-4289-805f-acc059ddc7b6 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:09.021479 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 202/429] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Sep 12 14:00:08 2022] GET /identity/v3/groups/2378a09b5ad64b1eb9d19d3c3d05296a => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:09.021803 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c22156-f954-4289-805f-acc059ddc7b6 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:09.042030 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-663643fb-5d80-4918-90e9-2014bc17bbef None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:09.044020 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-663643fb-5d80-4918-90e9-2014bc17bbef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:09.115222 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:09.124948 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 228/430] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:09.126173 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663643fb-5d80-4918-90e9-2014bc17bbef None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:09.150096 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:09.172533 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c22156-f954-4289-805f-acc059ddc7b6 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:09.243438 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b48523-f465-41e7-b650-062f97f9f784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:09.245745 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b48523-f465-41e7-b650-062f97f9f784 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:09.246226 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b48523-f465-41e7-b650-062f97f9f784 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:09.246869 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b48523-f465-41e7-b650-062f97f9f784 None admin] PATH_INFO: `/v3/groups/2378a09b5ad64b1eb9d19d3c3d05296a` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:09.249086 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b48523-f465-41e7-b650-062f97f9f784 None admin] RBAC: Authorizing `identity:get_group(group_id=2378a09b5ad64b1eb9d19d3c3d05296a)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:09.255020 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b48523-f465-41e7-b650-062f97f9f784 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:09.257913 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 203/431] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Sep 12 14:00:09 2022] GET /identity/v3/groups/2378a09b5ad64b1eb9d19d3c3d05296a => generated 226 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:09.280871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b48523-f465-41e7-b650-062f97f9f784 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:09.308220 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663643fb-5d80-4918-90e9-2014bc17bbef None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:09.363319 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc6aac-9d0f-4d54-8a67-7d98b0ff10ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:09.364275 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc6aac-9d0f-4d54-8a67-7d98b0ff10ff None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:09.364795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc6aac-9d0f-4d54-8a67-7d98b0ff10ff None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:09.365362 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc6aac-9d0f-4d54-8a67-7d98b0ff10ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:09.366308 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc6aac-9d0f-4d54-8a67-7d98b0ff10ff None admin] RBAC: Adding query filter params (group.id=2378a09b5ad64b1eb9d19d3c3d05296a, role.id=bcde1b76af8048deb2829c5aaa27296c, scope.project.id=e10c65c7a17c44a7a8711e7701efdcd0) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:09.366814 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc6aac-9d0f-4d54-8a67-7d98b0ff10ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:09.369224 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc6aac-9d0f-4d54-8a67-7d98b0ff10ff None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:09.375565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 229/432] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Sep 12 14:00:09 2022] GET /identity/v3/role_assignments?group.id=2378a09b5ad64b1eb9d19d3c3d05296a&scope.project.id=e10c65c7a17c44a7a8711e7701efdcd0&role.id=bcde1b76af8048deb2829c5aaa27296c => generated 611 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:09.409071 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc6aac-9d0f-4d54-8a67-7d98b0ff10ff None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:09.496912 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b48523-f465-41e7-b650-062f97f9f784 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:09.513036 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:09.562788 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc6aac-9d0f-4d54-8a67-7d98b0ff10ff None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:09.575354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc36ad44-a02f-40ca-8728-4c0d74e9ba56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:09.575963 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36ad44-a02f-40ca-8728-4c0d74e9ba56 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:09.576234 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36ad44-a02f-40ca-8728-4c0d74e9ba56 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:09.576498 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36ad44-a02f-40ca-8728-4c0d74e9ba56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:09.577223 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc36ad44-a02f-40ca-8728-4c0d74e9ba56 None admin] RBAC: Adding query filter params (group.id=2378a09b5ad64b1eb9d19d3c3d05296a, role.id=d0a6f7c5c34a4925b9efb7dd4ac915fc, scope.project.id=e10c65c7a17c44a7a8711e7701efdcd0) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:09.577521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc36ad44-a02f-40ca-8728-4c0d74e9ba56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:09.578964 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc36ad44-a02f-40ca-8728-4c0d74e9ba56 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:09.586138 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 204/433] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Sep 12 14:00:09 2022] GET /identity/v3/role_assignments?group.id=2378a09b5ad64b1eb9d19d3c3d05296a&scope.project.id=e10c65c7a17c44a7a8711e7701efdcd0&role.id=d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 611 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:00:09.620201 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583ff0c-184d-4daf-8645-991783756b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:09.621115 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583ff0c-184d-4daf-8645-991783756b2a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:09.621541 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583ff0c-184d-4daf-8645-991783756b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:09.621955 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583ff0c-184d-4daf-8645-991783756b2a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:09.640886 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3583ff0c-184d-4daf-8645-991783756b2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:09.646178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-3583ff0c-184d-4daf-8645-991783756b2a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 14:00:09.646178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:09.646178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:09.646178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:09.646178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:09.646178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:09.646178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:09.646178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:09.646178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:09.646178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:09.646178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:09.646178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:09.646178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:09.646178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:09.646178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:09.646178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:09.647335 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:09.647335 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:09.647335 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:09.647335 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:09.647335 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:09.647335 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:09.647335 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:09.647335 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:09.647335 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:09.647335 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:09.647335 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:09.647335 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:09.647335 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:09.647335 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:09.647335 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:09.647335 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:09.648378 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:09.648378 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:09.648378 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:09.648378 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:09.648378 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:09.648378 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:09.648378 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:09.648378 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:09.648378 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:09.648378 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:09.648378 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 14:00:09.648378 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:09.651026 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 230/434] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 14:00:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:00:09.660370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc36ad44-a02f-40ca-8728-4c0d74e9ba56 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:09.703508 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:09.814490 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc36ad44-a02f-40ca-8728-4c0d74e9ba56 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:09.850985 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8dd523-9a50-4998-b86a-8a3a9a313f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:09.851599 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8dd523-9a50-4998-b86a-8a3a9a313f9b None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:09.851956 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8dd523-9a50-4998-b86a-8a3a9a313f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:09.852218 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8dd523-9a50-4998-b86a-8a3a9a313f9b None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:09.852929 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8dd523-9a50-4998-b86a-8a3a9a313f9b None admin] RBAC: Adding query filter params (name=member) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:09.853230 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8dd523-9a50-4998-b86a-8a3a9a313f9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:09.854593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8dd523-9a50-4998-b86a-8a3a9a313f9b None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:09.863225 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 205/435] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 12 14:00:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:09.867176 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32a4f9-eacf-4fb7-bc78-12f183378bcf None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:09.867685 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32a4f9-eacf-4fb7-bc78-12f183378bcf None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:09.868971 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32a4f9-eacf-4fb7-bc78-12f183378bcf None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:09.869604 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8dd523-9a50-4998-b86a-8a3a9a313f9b None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:09.871446 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 231/436] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:09.878759 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201eadcb-d30c-4b2f-b027-5da29fd91ad3 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:09.879539 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201eadcb-d30c-4b2f-b027-5da29fd91ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:09.879821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201eadcb-d30c-4b2f-b027-5da29fd91ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:09.908612 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-201eadcb-d30c-4b2f-b027-5da29fd91ad3 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:09.910878 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-201eadcb-d30c-4b2f-b027-5da29fd91ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:09.970187 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:09.989785 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8dd523-9a50-4998-b86a-8a3a9a313f9b None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:09.994898 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 232/437] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:10.030173 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a648a1dd-bc39-4619-8b04-74109f98d910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:10.030883 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648a1dd-bc39-4619-8b04-74109f98d910 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:10.031146 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648a1dd-bc39-4619-8b04-74109f98d910 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:10.031515 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648a1dd-bc39-4619-8b04-74109f98d910 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:10.033012 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6af1f4-b213-4ffa-b31c-0fca5e1db73c None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:10.033299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6af1f4-b213-4ffa-b31c-0fca5e1db73c None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:10.033575 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6af1f4-b213-4ffa-b31c-0fca5e1db73c None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:10.036309 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 233/438] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 14:00:10.042517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9612999-2853-47a5-8734-31fbc1a3a83c None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:10.042802 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9612999-2853-47a5-8734-31fbc1a3a83c None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:10.043590 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9612999-2853-47a5-8734-31fbc1a3a83c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:10.067717 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-c9612999-2853-47a5-8734-31fbc1a3a83c None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:10.068856 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c9612999-2853-47a5-8734-31fbc1a3a83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:10.107714 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a648a1dd-bc39-4619-8b04-74109f98d910 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:10.109175 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a648a1dd-bc39-4619-8b04-74109f98d910 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:10.124927 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:10.150375 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 234/439] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:10.203785 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:10.219828 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 206/440] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:00:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 12 14:00:10.225530 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9612999-2853-47a5-8734-31fbc1a3a83c None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:10.281182 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a648a1dd-bc39-4619-8b04-74109f98d910 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:10.351446 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9612999-2853-47a5-8734-31fbc1a3a83c None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:10.394403 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac3d593-82ce-4254-93a7-7c31defe001b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:10.395401 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac3d593-82ce-4254-93a7-7c31defe001b None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:10.395947 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac3d593-82ce-4254-93a7-7c31defe001b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:10.396224 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac3d593-82ce-4254-93a7-7c31defe001b None admin] PATH_INFO: `/v3/projects/6606f4e0816344eca4228a4e9d0ed52e` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:10.397460 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac3d593-82ce-4254-93a7-7c31defe001b None admin] RBAC: Authorizing `identity:get_project(project_id=6606f4e0816344eca4228a4e9d0ed52e)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:10.399477 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac3d593-82ce-4254-93a7-7c31defe001b None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:10.400388 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a648a1dd-bc39-4619-8b04-74109f98d910 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:10.403051 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 235/441] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 14:00:10 2022] GET /identity/v3/projects/6606f4e0816344eca4228a4e9d0ed52e => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:10.406429 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac3d593-82ce-4254-93a7-7c31defe001b None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:10.440445 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02261ead-6a66-4a94-8606-306da6fb3156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:10.441074 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02261ead-6a66-4a94-8606-306da6fb3156 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:10.441353 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02261ead-6a66-4a94-8606-306da6fb3156 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:10.441624 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02261ead-6a66-4a94-8606-306da6fb3156 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:10.451921 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02261ead-6a66-4a94-8606-306da6fb3156 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:10.456910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-02261ead-6a66-4a94-8606-306da6fb3156 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:00:10.456910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:10.456910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:10.456910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:10.456910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:10.456910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:10.456910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:10.456910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:10.456910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:10.456910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:10.456910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:10.456910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:10.456910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:10.456910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:10.456910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:10.456910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:10.457887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:10.457887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:10.457887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:10.457887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:10.457887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:10.457887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:10.457887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:10.457887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:10.457887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:10.457887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:10.457887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:10.457887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:10.457887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:10.457887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:10.457887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:10.457887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:10.458661 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:10.458661 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:10.458661 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:10.458661 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:10.458661 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:10.458661 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:10.458661 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:10.458661 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:10.458661 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:10.458661 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:10.458661 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:00:10.458661 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:10.459424 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:10.459504 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 207/442] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 12 14:00:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:10.459504 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02261ead-6a66-4a94-8606-306da6fb3156 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:10.505836 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:10.510625 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac3d593-82ce-4254-93a7-7c31defe001b None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:10.545089 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723e52d3-3d0b-4040-8af2-44b40c5d72fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:10.545706 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e52d3-3d0b-4040-8af2-44b40c5d72fd None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:10.545980 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e52d3-3d0b-4040-8af2-44b40c5d72fd None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:10.546247 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e52d3-3d0b-4040-8af2-44b40c5d72fd None admin] PATH_INFO: `/v3/groups/2378a09b5ad64b1eb9d19d3c3d05296a` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:10.547366 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723e52d3-3d0b-4040-8af2-44b40c5d72fd None admin] RBAC: Authorizing `identity:get_group(group_id=2378a09b5ad64b1eb9d19d3c3d05296a)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:10.548774 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723e52d3-3d0b-4040-8af2-44b40c5d72fd None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:10.550312 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 236/443] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Sep 12 14:00:10 2022] GET /identity/v3/groups/2378a09b5ad64b1eb9d19d3c3d05296a => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:10.551079 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723e52d3-3d0b-4040-8af2-44b40c5d72fd None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:10.560461 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02261ead-6a66-4a94-8606-306da6fb3156 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:10.593974 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de75300-f940-4f71-be1a-c45e89e65d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:10.594592 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de75300-f940-4f71-be1a-c45e89e65d88 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:10.594884 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de75300-f940-4f71-be1a-c45e89e65d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:10.595149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de75300-f940-4f71-be1a-c45e89e65d88 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:10.605408 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de75300-f940-4f71-be1a-c45e89e65d88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:10.610242 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-6de75300-f940-4f71-be1a-c45e89e65d88 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 14:00:10.610242 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:10.610242 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:10.610242 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:10.610242 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:10.610242 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:10.610242 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:10.610242 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:10.610242 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:10.610242 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:10.610242 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:10.610242 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:10.610242 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:10.610242 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:10.610242 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:10.610242 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:10.611370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:10.611370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:10.611370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:10.611370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:10.611370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:10.611370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:10.611370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:10.611370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:10.611370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:10.611370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:10.611370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:10.611370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:10.611370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:10.611370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:10.611370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:10.611370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:10.612435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:10.612435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:10.612435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:10.612435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:10.612435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:10.612435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:10.612435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:10.612435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:10.612435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:10.612435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:10.612435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 14:00:10.612435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:10.612435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 208/444] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 12 14:00:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:10.612435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de75300-f940-4f71-be1a-c45e89e65d88 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:10.651916 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723e52d3-3d0b-4040-8af2-44b40c5d72fd None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:10.667980 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:10.685104 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae77d0-c65e-452c-b267-f0564792d44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:10.685702 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae77d0-c65e-452c-b267-f0564792d44a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:10.685985 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae77d0-c65e-452c-b267-f0564792d44a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:10.686254 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae77d0-c65e-452c-b267-f0564792d44a None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:10.686942 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ae77d0-c65e-452c-b267-f0564792d44a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:10.687244 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ae77d0-c65e-452c-b267-f0564792d44a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:10.688585 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ae77d0-c65e-452c-b267-f0564792d44a None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:10.696372 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 237/445] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 14:00:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:10.697420 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae77d0-c65e-452c-b267-f0564792d44a None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:10.713723 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de75300-f940-4f71-be1a-c45e89e65d88 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:10.720797 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:10.747390 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b37ebd-dc4c-4c8c-a4ab-f09184d5b439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:10.747980 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b37ebd-dc4c-4c8c-a4ab-f09184d5b439 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:10.748252 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b37ebd-dc4c-4c8c-a4ab-f09184d5b439 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:10.748516 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b37ebd-dc4c-4c8c-a4ab-f09184d5b439 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:10.749228 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b37ebd-dc4c-4c8c-a4ab-f09184d5b439 None admin] RBAC: Adding query filter params (group.id=2378a09b5ad64b1eb9d19d3c3d05296a, role.id=d0a6f7c5c34a4925b9efb7dd4ac915fc, scope.project.id=6606f4e0816344eca4228a4e9d0ed52e) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:10.749526 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b37ebd-dc4c-4c8c-a4ab-f09184d5b439 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:10.750966 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b37ebd-dc4c-4c8c-a4ab-f09184d5b439 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:10.756579 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 209/446] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Sep 12 14:00:10 2022] GET /identity/v3/role_assignments?group.id=2378a09b5ad64b1eb9d19d3c3d05296a&scope.project.id=6606f4e0816344eca4228a4e9d0ed52e&role.id=d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:10.757184 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b37ebd-dc4c-4c8c-a4ab-f09184d5b439 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:10.803836 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae77d0-c65e-452c-b267-f0564792d44a None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:10.841926 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09cc7c3-f90c-4b38-9ecc-540008fad6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:10.842585 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09cc7c3-f90c-4b38-9ecc-540008fad6fa None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:10.842826 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09cc7c3-f90c-4b38-9ecc-540008fad6fa None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:10.843124 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09cc7c3-f90c-4b38-9ecc-540008fad6fa None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:10.843805 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09cc7c3-f90c-4b38-9ecc-540008fad6fa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:10.844108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09cc7c3-f90c-4b38-9ecc-540008fad6fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:10.845494 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09cc7c3-f90c-4b38-9ecc-540008fad6fa None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:10.855360 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 238/447] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:00:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:10.861484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09cc7c3-f90c-4b38-9ecc-540008fad6fa None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:10.895319 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b37ebd-dc4c-4c8c-a4ab-f09184d5b439 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:10.910965 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:10.940870 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40b1eb-62bb-4f42-a45b-604bbbd563b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:10.941482 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40b1eb-62bb-4f42-a45b-604bbbd563b2 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:10.941758 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40b1eb-62bb-4f42-a45b-604bbbd563b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:10.943036 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40b1eb-62bb-4f42-a45b-604bbbd563b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:10.981014 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09cc7c3-f90c-4b38-9ecc-540008fad6fa None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:10.987380 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:11.018673 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369de2a4-5e71-4887-8c09-b9b26365adec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:11.018673 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369de2a4-5e71-4887-8c09-b9b26365adec None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:11.020852 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369de2a4-5e71-4887-8c09-b9b26365adec None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:11.020852 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369de2a4-5e71-4887-8c09-b9b26365adec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:11.024077 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e40b1eb-62bb-4f42-a45b-604bbbd563b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:11.026148 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e40b1eb-62bb-4f42-a45b-604bbbd563b2 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:11.093068 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369de2a4-5e71-4887-8c09-b9b26365adec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:11.094504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369de2a4-5e71-4887-8c09-b9b26365adec None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:11.141244 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:11.145936 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 210/448] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:00:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 14:00:11.152782 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40b1eb-62bb-4f42-a45b-604bbbd563b2 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:11.199133 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 239/449] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:00:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 14:00:11.204747 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369de2a4-5e71-4887-8c09-b9b26365adec None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:11.219979 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:11.250213 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40b1eb-62bb-4f42-a45b-604bbbd563b2 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:11.284846 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585585c-1a5f-468a-bdbb-b2538b072443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:11.285460 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585585c-1a5f-468a-bdbb-b2538b072443 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:11.285735 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585585c-1a5f-468a-bdbb-b2538b072443 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:11.286002 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585585c-1a5f-468a-bdbb-b2538b072443 None admin] PATH_INFO: `/v3/projects/b159fcc4f05d45de88c679e235640813` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:11.287133 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585585c-1a5f-468a-bdbb-b2538b072443 None admin] RBAC: Authorizing `identity:get_project(project_id=b159fcc4f05d45de88c679e235640813)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:11.288553 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585585c-1a5f-468a-bdbb-b2538b072443 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:11.294564 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 211/450] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 14:00:11 2022] GET /identity/v3/projects/b159fcc4f05d45de88c679e235640813 => generated 344 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:00:11.295559 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585585c-1a5f-468a-bdbb-b2538b072443 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:11.301832 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369de2a4-5e71-4887-8c09-b9b26365adec None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:11.335759 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31785936-d32c-4253-bdf2-5c7852bdeb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:11.336384 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31785936-d32c-4253-bdf2-5c7852bdeb55 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:11.336660 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31785936-d32c-4253-bdf2-5c7852bdeb55 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:11.336928 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31785936-d32c-4253-bdf2-5c7852bdeb55 None admin] PATH_INFO: `/v3/projects/6606f4e0816344eca4228a4e9d0ed52e` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:11.338097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31785936-d32c-4253-bdf2-5c7852bdeb55 None admin] RBAC: Authorizing `identity:get_project(project_id=6606f4e0816344eca4228a4e9d0ed52e)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:11.339532 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31785936-d32c-4253-bdf2-5c7852bdeb55 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:11.341137 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 240/451] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 14:00:11 2022] GET /identity/v3/projects/6606f4e0816344eca4228a4e9d0ed52e => generated 302 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:00:11.346508 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31785936-d32c-4253-bdf2-5c7852bdeb55 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:11.356317 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:11.393853 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585585c-1a5f-468a-bdbb-b2538b072443 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:11.427751 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b640013-789c-4e65-9302-3f18aae7d152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:11.428371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b640013-789c-4e65-9302-3f18aae7d152 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:11.428646 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b640013-789c-4e65-9302-3f18aae7d152 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:11.428918 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b640013-789c-4e65-9302-3f18aae7d152 None admin] PATH_INFO: `/v3/groups/e06ac03aadb7402884a9f12c228d338a` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:11.430010 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b640013-789c-4e65-9302-3f18aae7d152 None admin] RBAC: Authorizing `identity:get_group(group_id=e06ac03aadb7402884a9f12c228d338a)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:11.431991 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b640013-789c-4e65-9302-3f18aae7d152 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:11.433540 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 212/452] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Sep 12 14:00:11 2022] GET /identity/v3/groups/e06ac03aadb7402884a9f12c228d338a => generated 229 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:00:11.439218 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b640013-789c-4e65-9302-3f18aae7d152 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:11.447895 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:11.450046 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31785936-d32c-4253-bdf2-5c7852bdeb55 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:11.482768 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807831d5-09a2-43d9-8d41-c26ced8f8e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:11.483409 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807831d5-09a2-43d9-8d41-c26ced8f8e3c None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:11.483681 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807831d5-09a2-43d9-8d41-c26ced8f8e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:11.483946 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807831d5-09a2-43d9-8d41-c26ced8f8e3c None admin] PATH_INFO: `/v3/groups/2378a09b5ad64b1eb9d19d3c3d05296a` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:11.485056 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807831d5-09a2-43d9-8d41-c26ced8f8e3c None admin] RBAC: Authorizing `identity:get_group(group_id=2378a09b5ad64b1eb9d19d3c3d05296a)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:11.486468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807831d5-09a2-43d9-8d41-c26ced8f8e3c None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:11.488044 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 241/453] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Sep 12 14:00:11 2022] GET /identity/v3/groups/2378a09b5ad64b1eb9d19d3c3d05296a => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:00:11.494283 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807831d5-09a2-43d9-8d41-c26ced8f8e3c None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:11.537561 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b640013-789c-4e65-9302-3f18aae7d152 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:11.568143 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:11.570214 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee8181-6fab-4fc3-9f6e-7cdd61e5e2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:11.570815 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee8181-6fab-4fc3-9f6e-7cdd61e5e2a4 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:11.571221 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee8181-6fab-4fc3-9f6e-7cdd61e5e2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:11.571394 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee8181-6fab-4fc3-9f6e-7cdd61e5e2a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:11.572659 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fee8181-6fab-4fc3-9f6e-7cdd61e5e2a4 None admin] RBAC: Adding query filter params (group.id=e06ac03aadb7402884a9f12c228d338a, role.id=e14564f00daf48a28d4e32da8cd2efad, scope.project.id=b159fcc4f05d45de88c679e235640813) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:11.572962 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fee8181-6fab-4fc3-9f6e-7cdd61e5e2a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:11.574425 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fee8181-6fab-4fc3-9f6e-7cdd61e5e2a4 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:11.594866 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 213/454] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Sep 12 14:00:11 2022] GET /identity/v3/role_assignments?group.id=e06ac03aadb7402884a9f12c228d338a&scope.project.id=b159fcc4f05d45de88c679e235640813&role.id=e14564f00daf48a28d4e32da8cd2efad => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:00:11.595076 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807831d5-09a2-43d9-8d41-c26ced8f8e3c None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:11.627541 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae0631b-25a6-4008-8a5e-05b45efc6363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:11.628148 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae0631b-25a6-4008-8a5e-05b45efc6363 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:11.628422 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae0631b-25a6-4008-8a5e-05b45efc6363 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:11.628690 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae0631b-25a6-4008-8a5e-05b45efc6363 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:11.629425 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae0631b-25a6-4008-8a5e-05b45efc6363 None admin] RBAC: Adding query filter params (group.id=2378a09b5ad64b1eb9d19d3c3d05296a, role.id=bcde1b76af8048deb2829c5aaa27296c, scope.project.id=6606f4e0816344eca4228a4e9d0ed52e) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:11.629724 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae0631b-25a6-4008-8a5e-05b45efc6363 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:11.631195 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae0631b-25a6-4008-8a5e-05b45efc6363 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:11.639972 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 242/455] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Sep 12 14:00:11 2022] GET /identity/v3/role_assignments?group.id=2378a09b5ad64b1eb9d19d3c3d05296a&scope.project.id=6606f4e0816344eca4228a4e9d0ed52e&role.id=bcde1b76af8048deb2829c5aaa27296c => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:00:14.428337 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795f6cb-0d69-4eee-a3dd-e9fa2d6098ee None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:14.429332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795f6cb-0d69-4eee-a3dd-e9fa2d6098ee None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:14.429827 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795f6cb-0d69-4eee-a3dd-e9fa2d6098ee None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:14.431657 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 214/456] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:14.437409 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d5661-fbbf-4047-96bd-86be32b01ad0 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:14.438235 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d5661-fbbf-4047-96bd-86be32b01ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:14.438698 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d5661-fbbf-4047-96bd-86be32b01ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:14.463791 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-ea5d5661-fbbf-4047-96bd-86be32b01ad0 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:14.465302 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ea5d5661-fbbf-4047-96bd-86be32b01ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:14.562360 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 243/457] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 14:00:14.617917 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d777311-803d-4979-b487-6abb4e2a7d5d None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:14.618541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d777311-803d-4979-b487-6abb4e2a7d5d None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:14.619058 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d777311-803d-4979-b487-6abb4e2a7d5d None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:14.620812 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 215/458] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:14.625869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c30110-5432-471d-8a4f-23c6a19f51f8 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:14.626437 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c30110-5432-471d-8a4f-23c6a19f51f8 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:14.626950 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c30110-5432-471d-8a4f-23c6a19f51f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:14.648561 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-b6c30110-5432-471d-8a4f-23c6a19f51f8 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:14.650591 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b6c30110-5432-471d-8a4f-23c6a19f51f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:14.732154 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:14.736771 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 244/459] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:14.927478 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d777311-803d-4979-b487-6abb4e2a7d5d None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:15.019059 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:15.063053 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d777311-803d-4979-b487-6abb4e2a7d5d None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:15.111691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e9b02-edb2-43ca-8ee8-2aeb3505d8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:15.112681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e9b02-edb2-43ca-8ee8-2aeb3505d8ed None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:15.113149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e9b02-edb2-43ca-8ee8-2aeb3505d8ed None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:15.113638 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e9b02-edb2-43ca-8ee8-2aeb3505d8ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:15.118666 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c30110-5432-471d-8a4f-23c6a19f51f8 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:15.125429 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481e9b02-edb2-43ca-8ee8-2aeb3505d8ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:15.126888 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481e9b02-edb2-43ca-8ee8-2aeb3505d8ed None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:15.142964 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-481e9b02-edb2-43ca-8ee8-2aeb3505d8ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:15.142964 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:15.142964 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:15.142964 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:15.142964 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:15.142964 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:15.142964 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:15.142964 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:15.142964 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:15.142964 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:15.142964 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:15.142964 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:15.142964 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:15.142964 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:15.142964 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:15.142964 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:15.143910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:15.143910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:15.143910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:15.143910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:15.143910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:15.143910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:15.143910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:15.143910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:15.143910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:15.143910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:15.143910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:15.143910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:15.143910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:15.143910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:15.143910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:15.143910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:15.143910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:15.144762 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:15.144762 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:15.144762 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:15.144762 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 216/460] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:15.145689 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e2afe-8199-4efd-8a43-bd02e71fba52 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:15.145971 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e2afe-8199-4efd-8a43-bd02e71fba52 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:15.146241 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e2afe-8199-4efd-8a43-bd02e71fba52 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:15.147472 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 217/461] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:15.151701 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246e2afe-8199-4efd-8a43-bd02e71fba52 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:15.234133 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c30110-5432-471d-8a4f-23c6a19f51f8 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:15.259798 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246e2afe-8199-4efd-8a43-bd02e71fba52 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:15.277829 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a415609-2066-4719-a488-38896f3c699f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:15.278395 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a415609-2066-4719-a488-38896f3c699f None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:15.278677 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a415609-2066-4719-a488-38896f3c699f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:15.279052 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a415609-2066-4719-a488-38896f3c699f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:15.282256 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:15.293359 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a415609-2066-4719-a488-38896f3c699f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:15.294805 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a415609-2066-4719-a488-38896f3c699f None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:15.295233 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf45901b-dda7-4211-b6aa-140ce33ab53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:15.295884 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45901b-dda7-4211-b6aa-140ce33ab53d None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:15.296168 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45901b-dda7-4211-b6aa-140ce33ab53d None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:15.296442 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45901b-dda7-4211-b6aa-140ce33ab53d None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:15.297079 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf45901b-dda7-4211-b6aa-140ce33ab53d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:15.297379 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf45901b-dda7-4211-b6aa-140ce33ab53d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:15.298727 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf45901b-dda7-4211-b6aa-140ce33ab53d None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:15.310870 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 218/462] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:15.312197 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb92be9-f4a4-453f-a790-927dbfe74b48 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:15.312484 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb92be9-f4a4-453f-a790-927dbfe74b48 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:15.312757 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb92be9-f4a4-453f-a790-927dbfe74b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:15.313508 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-0a415609-2066-4719-a488-38896f3c699f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:15.313508 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:15.313508 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:15.313508 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:15.313508 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:15.313508 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:15.313508 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:15.313508 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:15.313508 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:15.313508 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:15.313508 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:15.313508 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:15.313508 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:15.313508 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:15.313508 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:15.313508 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:15.314455 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:15.314455 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:15.314455 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:15.314455 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:15.314455 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:15.314455 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:15.314455 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:15.314455 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:15.314455 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:15.314455 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:15.314455 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:15.314455 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:15.314455 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:15.314455 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:15.314455 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:15.314455 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:15.314455 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:15.315321 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:15.315321 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:15.315321 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:15.317008 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 245/463] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:15.318004 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be5430-14fe-4b08-a4c9-55087499db37 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:15.318287 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be5430-14fe-4b08-a4c9-55087499db37 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:15.318565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be5430-14fe-4b08-a4c9-55087499db37 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:15.320078 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 246/464] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:15.320778 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01be5430-14fe-4b08-a4c9-55087499db37 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:15.335368 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-5bb92be9-f4a4-453f-a790-927dbfe74b48 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:15.336561 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-5bb92be9-f4a4-453f-a790-927dbfe74b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:15.381522 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:15.402464 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 219/465] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:15.403539 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb92be9-f4a4-453f-a790-927dbfe74b48 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:15.443545 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01be5430-14fe-4b08-a4c9-55087499db37 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:15.482555 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e1b85f-4c26-439f-a475-d22e3e35d77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:15.483431 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1b85f-4c26-439f-a475-d22e3e35d77d None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:15.483853 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1b85f-4c26-439f-a475-d22e3e35d77d None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:15.484333 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1b85f-4c26-439f-a475-d22e3e35d77d None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:15.485204 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e1b85f-4c26-439f-a475-d22e3e35d77d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:15.486866 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e1b85f-4c26-439f-a475-d22e3e35d77d None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:15.514166 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 247/466] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Sep 12 14:00:15 2022] POST /identity/v3/users => generated 246 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:00:15.515951 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc5fc3-fbc3-4d6e-9a9b-4fae64a21fff None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:15.516441 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc5fc3-fbc3-4d6e-9a9b-4fae64a21fff None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:15.517006 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc5fc3-fbc3-4d6e-9a9b-4fae64a21fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:15.539839 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-7abc5fc3-fbc3-4d6e-9a9b-4fae64a21fff None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:15.541014 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb92be9-f4a4-453f-a790-927dbfe74b48 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:15.542055 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7abc5fc3-fbc3-4d6e-9a9b-4fae64a21fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:15.556530 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:15.584275 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb108aa-27fe-4173-af54-ee249e400fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:15.585037 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb108aa-27fe-4173-af54-ee249e400fd3 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:15.585371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb108aa-27fe-4173-af54-ee249e400fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:15.585648 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb108aa-27fe-4173-af54-ee249e400fd3 None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:15.586308 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb108aa-27fe-4173-af54-ee249e400fd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:15.586609 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb108aa-27fe-4173-af54-ee249e400fd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:15.589237 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb108aa-27fe-4173-af54-ee249e400fd3 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:15.608456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 220/467] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:15.610016 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb108aa-27fe-4173-af54-ee249e400fd3 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:15.726689 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 248/468] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:15.728278 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc5fc3-fbc3-4d6e-9a9b-4fae64a21fff None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:15.743490 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb108aa-27fe-4173-af54-ee249e400fd3 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:15.791446 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:15.796724 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1703cad-f6e8-44ea-b8fb-ab6fd35bdedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:15.797347 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1703cad-f6e8-44ea-b8fb-ab6fd35bdedc None admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:15.798112 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1703cad-f6e8-44ea-b8fb-ab6fd35bdedc None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:15.798390 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1703cad-f6e8-44ea-b8fb-ab6fd35bdedc None admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:15.799096 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1703cad-f6e8-44ea-b8fb-ab6fd35bdedc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:15.800923 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1703cad-f6e8-44ea-b8fb-ab6fd35bdedc None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:15.826991 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 221/469] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Sep 12 14:00:15 2022] POST /identity/v3/users => generated 248 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:00:15.882302 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc5fc3-fbc3-4d6e-9a9b-4fae64a21fff None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:15.904717 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5eae9-fec5-48fb-a3f0-ce8e68aaecc4 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:15.905041 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5eae9-fec5-48fb-a3f0-ce8e68aaecc4 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:15.905268 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5eae9-fec5-48fb-a3f0-ce8e68aaecc4 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:15.906935 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 222/470] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 14:00:15.912008 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a669e2-5758-45e0-81d4-d1341d5b8c3b None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:15.912285 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a669e2-5758-45e0-81d4-d1341d5b8c3b None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:15.912559 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a669e2-5758-45e0-81d4-d1341d5b8c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:15.936475 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473644e3-1817-4ada-898a-25eb9e4cb46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:15.937594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473644e3-1817-4ada-898a-25eb9e4cb46d None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:15.938108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473644e3-1817-4ada-898a-25eb9e4cb46d None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:15.938602 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473644e3-1817-4ada-898a-25eb9e4cb46d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:15.940418 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-97a669e2-5758-45e0-81d4-d1341d5b8c3b None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:15.941913 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-97a669e2-5758-45e0-81d4-d1341d5b8c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:15.952382 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473644e3-1817-4ada-898a-25eb9e4cb46d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:15.953858 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473644e3-1817-4ada-898a-25eb9e4cb46d None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:15.976347 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-473644e3-1817-4ada-898a-25eb9e4cb46d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:15.976347 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:15.976347 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:15.976347 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:15.976347 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:15.976347 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:15.976347 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:15.976347 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:15.976347 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:15.976347 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:15.976347 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:15.976347 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:15.976347 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:15.976347 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:15.976347 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:15.976347 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:15.977349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:15.977349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:15.977349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:15.977349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:15.977349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:15.977349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:15.977349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:15.977349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:15.977349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:15.977349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:15.977349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:15.977349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:15.977349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:15.977349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:15.977349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:15.977349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:15.977349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:15.978247 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:15.978247 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:15.978247 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:15.978408 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:15.980689 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 249/471] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:15.987487 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473644e3-1817-4ada-898a-25eb9e4cb46d None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:16.040459 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 223/472] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:16.076918 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a669e2-5758-45e0-81d4-d1341d5b8c3b None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:16.110132 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:16.123036 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473644e3-1817-4ada-898a-25eb9e4cb46d None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:16.161637 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b6e011-8c06-4678-9370-b75a0a31f73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:16.162239 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b6e011-8c06-4678-9370-b75a0a31f73a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:16.162515 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b6e011-8c06-4678-9370-b75a0a31f73a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:16.162782 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b6e011-8c06-4678-9370-b75a0a31f73a None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:16.163451 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b6e011-8c06-4678-9370-b75a0a31f73a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:16.163745 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b6e011-8c06-4678-9370-b75a0a31f73a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:16.165848 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b6e011-8c06-4678-9370-b75a0a31f73a None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:16.188153 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 250/473] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:16.194913 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b6e011-8c06-4678-9370-b75a0a31f73a None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:16.207125 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a669e2-5758-45e0-81d4-d1341d5b8c3b None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:16.258144 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c7512d-bf43-461f-b1e5-78a81b83b1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:16.259677 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7512d-bf43-461f-b1e5-78a81b83b1f8 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:16.260180 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7512d-bf43-461f-b1e5-78a81b83b1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:16.260657 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7512d-bf43-461f-b1e5-78a81b83b1f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:16.275802 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c7512d-bf43-461f-b1e5-78a81b83b1f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:16.278380 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c7512d-bf43-461f-b1e5-78a81b83b1f8 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:16.300542 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-e6c7512d-bf43-461f-b1e5-78a81b83b1f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:16.300542 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:16.300542 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:16.300542 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:16.300542 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:16.300542 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:16.300542 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:16.300542 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:16.300542 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:16.300542 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:16.300542 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:16.300542 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:16.300542 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:16.300542 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:16.300542 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:16.300542 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:16.301484 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:16.301484 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:16.301484 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:16.301484 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:16.301484 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:16.301484 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:16.301484 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:16.301484 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:16.301484 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:16.301484 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:16.301484 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:16.301484 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:16.301484 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:16.301484 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:16.301484 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:16.301484 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:16.301484 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:16.302316 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:16.302316 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:16.302316 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:16.304315 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 224/474] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:00:16.310803 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c7512d-bf43-461f-b1e5-78a81b83b1f8 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:16.324366 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b6e011-8c06-4678-9370-b75a0a31f73a None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:16.326098 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:16.378436 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d848a5-cbc6-4762-97f5-1d05dff08461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:16.379158 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d848a5-cbc6-4762-97f5-1d05dff08461 None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:16.379553 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d848a5-cbc6-4762-97f5-1d05dff08461 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:16.379927 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d848a5-cbc6-4762-97f5-1d05dff08461 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:16.380673 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d848a5-cbc6-4762-97f5-1d05dff08461 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:16.382181 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d848a5-cbc6-4762-97f5-1d05dff08461 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:16.407333 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 251/475] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Sep 12 14:00:16 2022] POST /identity/v3/users => generated 248 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:00:16.415013 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d848a5-cbc6-4762-97f5-1d05dff08461 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:16.428655 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:16.485184 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c7512d-bf43-461f-b1e5-78a81b83b1f8 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:16.532881 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c917b1b-9329-4fad-89ad-c382108c7a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:16.533734 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c917b1b-9329-4fad-89ad-c382108c7a58 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:16.534149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c917b1b-9329-4fad-89ad-c382108c7a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:16.534555 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c917b1b-9329-4fad-89ad-c382108c7a58 None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:16.535419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c917b1b-9329-4fad-89ad-c382108c7a58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:16.535879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c917b1b-9329-4fad-89ad-c382108c7a58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:16.537379 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c917b1b-9329-4fad-89ad-c382108c7a58 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:16.552493 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d848a5-cbc6-4762-97f5-1d05dff08461 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:16.560505 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 225/476] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:16.566107 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c917b1b-9329-4fad-89ad-c382108c7a58 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:16.606294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588ae9d-7cb1-42c0-8308-f53d44b9897b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:16.607267 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588ae9d-7cb1-42c0-8308-f53d44b9897b None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:16.607734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588ae9d-7cb1-42c0-8308-f53d44b9897b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:16.608168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588ae9d-7cb1-42c0-8308-f53d44b9897b None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:16.608995 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0588ae9d-7cb1-42c0-8308-f53d44b9897b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:16.610632 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0588ae9d-7cb1-42c0-8308-f53d44b9897b None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:16.629998 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.sql.core [None req-0588ae9d-7cb1-42c0-8308-f53d44b9897b None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 12 14:00:16.629998 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 12 14:00:16.629998 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [parameters: {'id': 'b3c6688861674584920c51f88f0c3b03', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 12 14:00:16.629998 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81597) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 12 14:00:16.650576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-0588ae9d-7cb1-42c0-8308-f53d44b9897b None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 12 14:00:16.650576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:16.650576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 12 14:00:16.650576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 12 14:00:16.650576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 12 14:00:16.650576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 12 14:00:16.650576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 12 14:00:16.650576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application result = self._query(query) Sep 12 14:00:16.650576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 12 14:00:16.650576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application conn.query(q) Sep 12 14:00:16.650576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 12 14:00:16.650576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 12 14:00:16.650576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 12 14:00:16.650576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application result.read() Sep 12 14:00:16.650576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 12 14:00:16.651773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 12 14:00:16.651773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 12 14:00:16.651773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 12 14:00:16.651773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 12 14:00:16.651773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 12 14:00:16.651773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 12 14:00:16.651773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 12 14:00:16.651773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 12 14:00:16.651773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:16.651773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 12 14:00:16.651773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:16.651773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:16.651773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 12 14:00:16.651773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 12 14:00:16.651773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 12 14:00:16.651773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return ref.to_dict() Sep 12 14:00:16.651773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 12 14:00:16.651773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application next(self.gen) Sep 12 14:00:16.652666 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 12 14:00:16.652666 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application yield resource Sep 12 14:00:16.652666 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 12 14:00:16.652666 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application next(self.gen) Sep 12 14:00:16.652666 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 12 14:00:16.652666 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self.session.rollback() Sep 12 14:00:16.652666 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 12 14:00:16.652666 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self.force_reraise() Sep 12 14:00:16.652666 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 12 14:00:16.652666 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise self.value Sep 12 14:00:16.652666 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 12 14:00:16.652666 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 12 14:00:16.652666 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 12 14:00:16.652666 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application session.commit() Sep 12 14:00:16.652666 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 12 14:00:16.652666 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 12 14:00:16.652666 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 12 14:00:16.653552 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._prepare_impl() Sep 12 14:00:16.653552 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 12 14:00:16.653552 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self.session.flush() Sep 12 14:00:16.653552 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 12 14:00:16.653552 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._flush(objects) Sep 12 14:00:16.653552 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 12 14:00:16.653552 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 12 14:00:16.653552 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 12 14:00:16.653552 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application compat.raise_( Sep 12 14:00:16.653552 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 12 14:00:16.653552 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception Sep 12 14:00:16.653552 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 12 14:00:16.653552 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application flush_context.execute() Sep 12 14:00:16.653552 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 12 14:00:16.653552 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rec.execute(self) Sep 12 14:00:16.653552 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 12 14:00:16.653552 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 12 14:00:16.654428 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 12 14:00:16.654428 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 12 14:00:16.654428 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 12 14:00:16.654428 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 12 14:00:16.654428 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 12 14:00:16.654428 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 12 14:00:16.654428 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 12 14:00:16.654428 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 12 14:00:16.654428 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 12 14:00:16.654428 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 12 14:00:16.654428 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 12 14:00:16.654428 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 12 14:00:16.654428 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 12 14:00:16.654428 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 12 14:00:16.654428 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 12 14:00:16.654428 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception Sep 12 14:00:16.656667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 12 14:00:16.656667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 12 14:00:16.656667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 12 14:00:16.656667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 12 14:00:16.656667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 12 14:00:16.656667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application result = self._query(query) Sep 12 14:00:16.656667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 12 14:00:16.656667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application conn.query(q) Sep 12 14:00:16.656667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 12 14:00:16.656667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 12 14:00:16.656667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 12 14:00:16.656667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application result.read() Sep 12 14:00:16.656667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 12 14:00:16.656667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 12 14:00:16.656667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 12 14:00:16.656667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 12 14:00:16.658636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 12 14:00:16.658636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 12 14:00:16.658636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 12 14:00:16.658636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 12 14:00:16.658636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 12 14:00:16.658636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 12 14:00:16.658636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [parameters: {'id': 'b3c6688861674584920c51f88f0c3b03', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 12 14:00:16.658636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 12 14:00:16.658636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:16.658636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 14:00:16.658636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:16.658636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:16.658636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:16.658636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:16.658636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:16.658636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:16.660641 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:16.660641 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:16.660641 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:16.660641 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:16.660641 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:16.660641 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:16.660641 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 12 14:00:16.660641 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 12 14:00:16.660641 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:16.660641 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:16.660641 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 12 14:00:16.660641 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 12 14:00:16.660641 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 12 14:00:16.660641 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 12 14:00:16.660641 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 12 14:00:16.660641 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:16.664170 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 252/477] 10.222.0.42 () {60 vars in 1131 bytes} [Mon Sep 12 14:00:16 2022] POST /identity/v3/roles => generated 144 bytes in 256 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:16.671117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588ae9d-7cb1-42c0-8308-f53d44b9897b None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:16.699120 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:16.747944 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c917b1b-9329-4fad-89ad-c382108c7a58 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:16.764587 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:16.800942 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b6d0b-a11d-4571-b540-f56296c373d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:16.801572 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b6d0b-a11d-4571-b540-f56296c373d0 None admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:16.801848 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b6d0b-a11d-4571-b540-f56296c373d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:16.802117 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b6d0b-a11d-4571-b540-f56296c373d0 None admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:16.803170 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161b6d0b-a11d-4571-b540-f56296c373d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:16.807548 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161b6d0b-a11d-4571-b540-f56296c373d0 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:16.809812 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588ae9d-7cb1-42c0-8308-f53d44b9897b None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:16.838073 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 226/478] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Sep 12 14:00:16 2022] POST /identity/v3/users => generated 249 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:00:16.866026 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d720360-eb05-480b-a795-2713f9c10076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:16.866990 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d720360-eb05-480b-a795-2713f9c10076 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:16.867538 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d720360-eb05-480b-a795-2713f9c10076 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:16.867953 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d720360-eb05-480b-a795-2713f9c10076 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:16.883192 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d720360-eb05-480b-a795-2713f9c10076 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:16.889991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-8d720360-eb05-480b-a795-2713f9c10076 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 14:00:16.889991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:16.889991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:16.889991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:16.889991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:16.889991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:16.889991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:16.889991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:16.889991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:16.889991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:16.889991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:16.889991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:16.889991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:16.889991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:16.889991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:16.889991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:16.891243 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:16.891243 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:16.891243 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:16.891243 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:16.891243 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:16.891243 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:16.891243 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:16.891243 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:16.891243 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:16.891243 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:16.891243 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:16.891243 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:16.891243 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:16.891243 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:16.891243 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:16.891243 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:16.892251 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:16.892251 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:16.892251 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:16.892251 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:16.892251 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:16.892251 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:16.892251 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:16.892251 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:16.892251 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:16.892251 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:16.892251 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 14:00:16.892251 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:16.894614 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 253/479] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 12 14:00:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:00:16.901602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b6d0b-a11d-4571-b540-f56296c373d0 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:17.020377 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b6d0b-a11d-4571-b540-f56296c373d0 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:17.051205 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:17.059140 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11eccb1-1ab6-4cfd-9c3a-ac8b646a7b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:17.060026 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11eccb1-1ab6-4cfd-9c3a-ac8b646a7b73 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:17.060454 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11eccb1-1ab6-4cfd-9c3a-ac8b646a7b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:17.060817 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11eccb1-1ab6-4cfd-9c3a-ac8b646a7b73 None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:17.061627 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11eccb1-1ab6-4cfd-9c3a-ac8b646a7b73 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:17.062017 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11eccb1-1ab6-4cfd-9c3a-ac8b646a7b73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:17.063547 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11eccb1-1ab6-4cfd-9c3a-ac8b646a7b73 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:17.075554 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 227/480] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:00:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:18.240056 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3eedc-5115-4624-aeeb-0232b3c0c2e3 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:18.241049 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3eedc-5115-4624-aeeb-0232b3c0c2e3 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:18.241539 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3eedc-5115-4624-aeeb-0232b3c0c2e3 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:18.243907 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 254/481] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:18.257216 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d1af4-04c5-4f36-a28e-aba86299c274 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:18.258572 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d1af4-04c5-4f36-a28e-aba86299c274 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:18.259639 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d1af4-04c5-4f36-a28e-aba86299c274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:18.283829 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-cb8d1af4-04c5-4f36-a28e-aba86299c274 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:18.285574 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-cb8d1af4-04c5-4f36-a28e-aba86299c274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:18.380374 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 228/482] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:18.735356 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3eedc-5115-4624-aeeb-0232b3c0c2e3 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:18.800203 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b50bc76-63b0-4f4c-92a7-c729b6f24ee6 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:18.800551 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b50bc76-63b0-4f4c-92a7-c729b6f24ee6 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:18.800784 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b50bc76-63b0-4f4c-92a7-c729b6f24ee6 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:18.802379 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 229/483] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:18.816485 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56092f-4bd4-4b6b-a92e-aa196d277a81 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:18.816760 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56092f-4bd4-4b6b-a92e-aa196d277a81 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:18.817017 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56092f-4bd4-4b6b-a92e-aa196d277a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:18.858281 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3eedc-5115-4624-aeeb-0232b3c0c2e3 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:18.859870 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-2c56092f-4bd4-4b6b-a92e-aa196d277a81 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:18.861899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-2c56092f-4bd4-4b6b-a92e-aa196d277a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:18.889030 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:18.895328 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6190b-75d4-4e94-95fb-8d347679fc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:18.896182 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6190b-75d4-4e94-95fb-8d347679fc7a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:18.896674 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6190b-75d4-4e94-95fb-8d347679fc7a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:18.897100 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6190b-75d4-4e94-95fb-8d347679fc7a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:18.909963 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f6190b-75d4-4e94-95fb-8d347679fc7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:18.915997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-60f6190b-75d4-4e94-95fb-8d347679fc7a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:00:18.915997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:18.915997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:18.915997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:18.915997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:18.915997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:18.915997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:18.915997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:18.915997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:18.915997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:18.915997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:18.915997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:18.915997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:18.915997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:18.915997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:18.915997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:18.916894 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:18.916894 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:18.916894 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:18.916894 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:18.916894 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:18.916894 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:18.916894 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:18.916894 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:18.916894 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:18.916894 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:18.916894 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:18.916894 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:18.916894 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:18.916894 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:18.916894 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:18.916894 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:18.917687 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:18.917687 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:18.917687 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:18.917687 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:18.917687 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:18.917687 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:18.917687 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:18.917687 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:18.917687 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:18.917687 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:18.917687 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:00:18.917687 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:18.920073 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 255/484] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 12 14:00:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:18.926884 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6190b-75d4-4e94-95fb-8d347679fc7a None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:19.016415 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 230/485] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:19.034963 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6190b-75d4-4e94-95fb-8d347679fc7a None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:19.070162 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc02848f-75ed-4a9b-a96d-677492660333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:19.071017 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc02848f-75ed-4a9b-a96d-677492660333 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:19.071472 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc02848f-75ed-4a9b-a96d-677492660333 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:19.072204 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc02848f-75ed-4a9b-a96d-677492660333 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:19.073004 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc02848f-75ed-4a9b-a96d-677492660333 None admin] RBAC: Adding query filter params (name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:19.073445 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc02848f-75ed-4a9b-a96d-677492660333 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:19.074965 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc02848f-75ed-4a9b-a96d-677492660333 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:19.083528 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 256/486] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 12 14:00:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:00:19.089869 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56092f-4bd4-4b6b-a92e-aa196d277a81 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:19.140039 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:19.214526 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56092f-4bd4-4b6b-a92e-aa196d277a81 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:19.259698 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d015b4-c71d-47b4-8999-bcc2b3d0b825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:19.261025 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d015b4-c71d-47b4-8999-bcc2b3d0b825 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:19.261456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d015b4-c71d-47b4-8999-bcc2b3d0b825 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:19.261866 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d015b4-c71d-47b4-8999-bcc2b3d0b825 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:19.277829 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d015b4-c71d-47b4-8999-bcc2b3d0b825 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:19.279525 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d015b4-c71d-47b4-8999-bcc2b3d0b825 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:19.298829 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-91d015b4-c71d-47b4-8999-bcc2b3d0b825 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:19.298829 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:19.298829 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:19.298829 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:19.298829 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:19.298829 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:19.298829 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:19.298829 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:19.298829 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:19.298829 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:19.298829 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:19.298829 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:19.298829 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:19.298829 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:19.298829 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:19.298829 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:19.299899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:19.299899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:19.299899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:19.299899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:19.299899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:19.299899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:19.299899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:19.299899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:19.299899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:19.299899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:19.299899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:19.299899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:19.299899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:19.299899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:19.299899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:19.299899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:19.299899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:19.300902 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:19.300902 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:19.300902 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:19.302272 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 231/487] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:19.308208 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc02848f-75ed-4a9b-a96d-677492660333 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:19.367318 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:19.387493 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d015b4-c71d-47b4-8999-bcc2b3d0b825 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:19.424864 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc02848f-75ed-4a9b-a96d-677492660333 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:19.461846 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dcc38c-7ae5-438e-b54e-80c7a9bf380c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:19.463228 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dcc38c-7ae5-438e-b54e-80c7a9bf380c None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:19.463666 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dcc38c-7ae5-438e-b54e-80c7a9bf380c None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:19.464070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dcc38c-7ae5-438e-b54e-80c7a9bf380c None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:19.464917 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dcc38c-7ae5-438e-b54e-80c7a9bf380c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:19.465507 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dcc38c-7ae5-438e-b54e-80c7a9bf380c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:19.467142 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dcc38c-7ae5-438e-b54e-80c7a9bf380c None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:19.481391 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 257/488] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:19.485873 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789edea4-c4cb-4d41-81e2-dc8b00cb4c4f None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:19.486319 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789edea4-c4cb-4d41-81e2-dc8b00cb4c4f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:19.486751 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789edea4-c4cb-4d41-81e2-dc8b00cb4c4f None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:19.488552 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 258/489] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 14:00:19.494867 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae36dda-dca5-4675-8495-5eec7e33ca1e None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:19.494867 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae36dda-dca5-4675-8495-5eec7e33ca1e None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:19.494867 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae36dda-dca5-4675-8495-5eec7e33ca1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:19.515253 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-4ae36dda-dca5-4675-8495-5eec7e33ca1e None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:19.516562 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-4ae36dda-dca5-4675-8495-5eec7e33ca1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:19.532011 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:19.535076 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d015b4-c71d-47b4-8999-bcc2b3d0b825 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:19.576670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f27e10-3496-416f-a748-be2f51b09662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:19.577474 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f27e10-3496-416f-a748-be2f51b09662 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:19.577849 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f27e10-3496-416f-a748-be2f51b09662 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:19.578299 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f27e10-3496-416f-a748-be2f51b09662 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:19.590004 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f27e10-3496-416f-a748-be2f51b09662 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:19.595505 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-d2f27e10-3496-416f-a748-be2f51b09662 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:00:19.595505 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:19.595505 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:19.595505 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:19.595505 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:19.595505 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:19.595505 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:19.595505 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:19.595505 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:19.595505 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:19.595505 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:19.595505 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:19.595505 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:19.595505 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:19.595505 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:19.595505 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:19.596541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:19.596541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:19.596541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:19.596541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:19.596541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:19.596541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:19.596541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:19.596541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:19.596541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:19.596541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:19.596541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:19.596541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:19.596541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:19.596541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:19.596541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:19.596541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:19.597455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:19.597455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:19.597455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:19.597455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:19.597455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:19.597455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:19.597455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:19.597455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:19.597455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:19.597455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:19.597455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:00:19.597455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:19.599467 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 232/490] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 12 14:00:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:00:19.600421 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f27e10-3496-416f-a748-be2f51b09662 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:19.607028 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 259/491] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:19.607958 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae36dda-dca5-4675-8495-5eec7e33ca1e None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:19.686725 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:19.754905 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae36dda-dca5-4675-8495-5eec7e33ca1e None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:19.755695 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f27e10-3496-416f-a748-be2f51b09662 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:19.796237 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1710d17-333c-469a-a154-a06401dc9ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:19.796927 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1710d17-333c-469a-a154-a06401dc9ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:19.797305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1710d17-333c-469a-a154-a06401dc9ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:19.797675 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1710d17-333c-469a-a154-a06401dc9ae6 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:19.798442 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1710d17-333c-469a-a154-a06401dc9ae6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:19.798864 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1710d17-333c-469a-a154-a06401dc9ae6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:19.799032 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6713a1-2bab-48ce-bb69-205076121b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:19.799661 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6713a1-2bab-48ce-bb69-205076121b13 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:19.799939 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6713a1-2bab-48ce-bb69-205076121b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:19.800307 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6713a1-2bab-48ce-bb69-205076121b13 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:19.800711 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1710d17-333c-469a-a154-a06401dc9ae6 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:19.809947 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6713a1-2bab-48ce-bb69-205076121b13 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:19.810730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 260/492] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 12 14:00:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:19.812103 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d97bd6-ed89-49f4-9846-fc04f093c923 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:19.812840 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d97bd6-ed89-49f4-9846-fc04f093c923 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:19.813213 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d97bd6-ed89-49f4-9846-fc04f093c923 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:19.814630 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 261/493] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:19.815720 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d97bd6-ed89-49f4-9846-fc04f093c923 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:19.815979 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6713a1-2bab-48ce-bb69-205076121b13 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:19.841580 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-6d6713a1-2bab-48ce-bb69-205076121b13 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 14:00:19.841580 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:19.841580 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:19.841580 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:19.841580 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:19.841580 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:19.841580 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:19.841580 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:19.841580 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:19.841580 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:19.841580 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:19.841580 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:19.841580 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:00:19.841580 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:00:19.841580 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:00:19.841580 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:00:19.842533 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:19.842533 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:19.842533 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:00:19.842533 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:19.842533 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:00:19.842533 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:19.842533 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:19.842533 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:19.842533 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:19.842533 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:19.842533 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:19.842533 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:19.842533 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:19.842533 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:19.842533 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:19.842533 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:19.842533 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:19.842533 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:19.844406 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:19.844406 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:19.844406 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:00:19.844406 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:00:19.844406 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:00:19.844406 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:00:19.844406 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:00:19.844406 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:00:19.844406 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 14:00:19.844406 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:19.844406 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 233/494] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Sep 12 14:00:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:19.847139 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9f2d6-597d-4c97-8bad-914c5bef3ad8 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:19.847643 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9f2d6-597d-4c97-8bad-914c5bef3ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:19.848115 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9f2d6-597d-4c97-8bad-914c5bef3ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:19.875828 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-d0c9f2d6-597d-4c97-8bad-914c5bef3ad8 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:19.877748 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-d0c9f2d6-597d-4c97-8bad-914c5bef3ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:19.895704 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:19.953116 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d97bd6-ed89-49f4-9846-fc04f093c923 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:19.953625 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 234/495] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:19.954792 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c9f2d6-597d-4c97-8bad-914c5bef3ad8 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:19.960147 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:19.994408 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931579c-cd1d-41b3-8f32-7947b9fca518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:19.995403 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931579c-cd1d-41b3-8f32-7947b9fca518 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:19.995860 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931579c-cd1d-41b3-8f32-7947b9fca518 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:19.996313 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931579c-cd1d-41b3-8f32-7947b9fca518 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:20.006472 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931579c-cd1d-41b3-8f32-7947b9fca518 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:20.007933 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931579c-cd1d-41b3-8f32-7947b9fca518 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:20.022058 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-1931579c-cd1d-41b3-8f32-7947b9fca518 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:20.022058 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:20.022058 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:20.022058 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:20.022058 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:20.022058 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:20.022058 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:20.022058 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:20.022058 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:20.022058 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:20.022058 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:20.022058 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:20.022058 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:20.022058 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:20.022058 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:20.022058 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:20.023173 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:20.023173 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:20.023173 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:20.023173 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:20.023173 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:20.023173 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:20.023173 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:20.023173 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:20.023173 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:20.023173 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:20.023173 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:20.023173 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:20.023173 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:20.023173 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:20.023173 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:20.023173 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:20.023173 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:20.024036 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:20.024036 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:20.024036 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:20.024036 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 262/496] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:20.024311 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931579c-cd1d-41b3-8f32-7947b9fca518 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:20.078256 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c9f2d6-597d-4c97-8bad-914c5bef3ad8 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:20.113311 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd0ee73-bc81-4e64-a0f8-b5eb010fa2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:20.113907 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd0ee73-bc81-4e64-a0f8-b5eb010fa2b5 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:20.114185 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd0ee73-bc81-4e64-a0f8-b5eb010fa2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:20.114487 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd0ee73-bc81-4e64-a0f8-b5eb010fa2b5 None admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:20.115205 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd0ee73-bc81-4e64-a0f8-b5eb010fa2b5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:20.115799 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd0ee73-bc81-4e64-a0f8-b5eb010fa2b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:20.116935 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd0ee73-bc81-4e64-a0f8-b5eb010fa2b5 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:20.135396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931579c-cd1d-41b3-8f32-7947b9fca518 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:20.158576 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 235/497] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Sep 12 14:00:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:20.158576 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb554e-f070-4f91-b95b-a978e837df36 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:20.158576 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb554e-f070-4f91-b95b-a978e837df36 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:20.158576 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb554e-f070-4f91-b95b-a978e837df36 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:20.158576 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 236/498] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:20.159429 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb554e-f070-4f91-b95b-a978e837df36 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:20.170788 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:20.175713 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d683a5e9-70ed-4003-9aa2-8ed755aa4dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:20.176348 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683a5e9-70ed-4003-9aa2-8ed755aa4dea None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:20.176626 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683a5e9-70ed-4003-9aa2-8ed755aa4dea None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:20.176896 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683a5e9-70ed-4003-9aa2-8ed755aa4dea None admin] PATH_INFO: `/v3/roles/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:20.187559 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d683a5e9-70ed-4003-9aa2-8ed755aa4dea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:20.192795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-d683a5e9-70ed-4003-9aa2-8ed755aa4dea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:00:20.192795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:20.192795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:20.192795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:20.192795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:20.192795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:20.192795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:20.192795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:20.192795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:20.192795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:20.192795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:20.192795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:20.192795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:20.192795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:20.192795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:20.192795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:20.193676 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:20.193676 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:20.193676 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:20.193676 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:20.193676 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:20.193676 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:20.193676 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:20.193676 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:20.193676 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:20.193676 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:20.193676 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:20.193676 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:20.193676 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:20.193676 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:20.193676 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:20.193676 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:20.194477 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:20.194477 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:20.194477 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:20.194477 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:20.194477 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:20.194477 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:20.194477 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:20.194477 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:20.194477 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:20.194477 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:20.194477 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:00:20.194477 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:20.194477 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 263/499] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 12 14:00:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:20.195195 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d683a5e9-70ed-4003-9aa2-8ed755aa4dea None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:20.217168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:20.272449 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb554e-f070-4f91-b95b-a978e837df36 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:20.299738 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d683a5e9-70ed-4003-9aa2-8ed755aa4dea None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:20.307319 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89f2fbe-2d94-416e-8d63-38aa17e3ec83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:20.309088 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f2fbe-2d94-416e-8d63-38aa17e3ec83 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:20.311098 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f2fbe-2d94-416e-8d63-38aa17e3ec83 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:20.311384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f2fbe-2d94-416e-8d63-38aa17e3ec83 None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:20.312060 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89f2fbe-2d94-416e-8d63-38aa17e3ec83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:20.312582 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89f2fbe-2d94-416e-8d63-38aa17e3ec83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:20.314384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89f2fbe-2d94-416e-8d63-38aa17e3ec83 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:20.325941 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 237/500] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:20.325941 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b2f4a-f713-4b8a-a51c-145a5c0a9ecb None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:20.325941 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b2f4a-f713-4b8a-a51c-145a5c0a9ecb None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:20.325941 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b2f4a-f713-4b8a-a51c-145a5c0a9ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:20.338211 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d33dafd-edcf-49e9-aa44-1da1d581864e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:20.338806 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d33dafd-edcf-49e9-aa44-1da1d581864e None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:20.339128 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d33dafd-edcf-49e9-aa44-1da1d581864e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:20.339449 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d33dafd-edcf-49e9-aa44-1da1d581864e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:20.345197 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-862b2f4a-f713-4b8a-a51c-145a5c0a9ecb None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:20.346375 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-862b2f4a-f713-4b8a-a51c-145a5c0a9ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:20.405197 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d33dafd-edcf-49e9-aa44-1da1d581864e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:20.406634 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d33dafd-edcf-49e9-aa44-1da1d581864e None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:20.406744 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:20.412251 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 238/501] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:20.413234 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b2f4a-f713-4b8a-a51c-145a5c0a9ecb None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:20.437905 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:20.516831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b2f4a-f713-4b8a-a51c-145a5c0a9ecb None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:20.524139 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 264/502] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:00:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 14:00:20.529220 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d33dafd-edcf-49e9-aa44-1da1d581864e None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:20.554270 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a75f640-abef-452b-9986-1fb9d7b1c2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:20.554914 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75f640-abef-452b-9986-1fb9d7b1c2d1 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:20.555212 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75f640-abef-452b-9986-1fb9d7b1c2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:20.555506 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75f640-abef-452b-9986-1fb9d7b1c2d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:20.556169 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a75f640-abef-452b-9986-1fb9d7b1c2d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:20.556485 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a75f640-abef-452b-9986-1fb9d7b1c2d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:20.557834 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a75f640-abef-452b-9986-1fb9d7b1c2d1 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:20.565676 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 239/503] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 12 14:00:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:20.566456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a75f640-abef-452b-9986-1fb9d7b1c2d1 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:20.627733 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:20.632776 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d33dafd-edcf-49e9-aa44-1da1d581864e None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:20.649829 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:20.670440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e41f68-646a-4cb9-afeb-f2c23e55b7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:20.671093 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e41f68-646a-4cb9-afeb-f2c23e55b7b7 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:20.671374 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e41f68-646a-4cb9-afeb-f2c23e55b7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:20.671656 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e41f68-646a-4cb9-afeb-f2c23e55b7b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:20.674691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a75f640-abef-452b-9986-1fb9d7b1c2d1 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:20.681761 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e41f68-646a-4cb9-afeb-f2c23e55b7b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:20.683227 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e41f68-646a-4cb9-afeb-f2c23e55b7b7 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:20.698504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-60e41f68-646a-4cb9-afeb-f2c23e55b7b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:20.698504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:20.698504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:20.698504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:20.698504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:20.698504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:20.698504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:20.698504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:20.698504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:20.698504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:20.698504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:20.698504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:20.698504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:20.698504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:20.698504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:20.698504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:20.699596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:20.699596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:20.699596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:20.699596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:20.699596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:20.699596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:20.699596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:20.699596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:20.699596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:20.699596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:20.699596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:20.699596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:20.699596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:20.699596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:20.699596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:20.699596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:20.699596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:20.700517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:20.700517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:20.700517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:20.700517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 265/504] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:20.700922 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e41f68-646a-4cb9-afeb-f2c23e55b7b7 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:20.710446 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68b4af-7efe-4b21-9cd6-b3d61d9cc8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:20.711097 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68b4af-7efe-4b21-9cd6-b3d61d9cc8fa None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:20.711373 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68b4af-7efe-4b21-9cd6-b3d61d9cc8fa None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:20.711665 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68b4af-7efe-4b21-9cd6-b3d61d9cc8fa None admin] PATH_INFO: `/v3/users/glance` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:20.719598 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c68b4af-7efe-4b21-9cd6-b3d61d9cc8fa None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:20.721015 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c68b4af-7efe-4b21-9cd6-b3d61d9cc8fa None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:20.731813 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-4c68b4af-7efe-4b21-9cd6-b3d61d9cc8fa None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 12 14:00:20.731813 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:20.731813 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:20.731813 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:20.731813 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:20.731813 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:20.731813 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:20.731813 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:20.731813 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:20.731813 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:20.731813 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:20.731813 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:20.731813 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:00:20.731813 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:00:20.731813 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:00:20.731813 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:00:20.732831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:20.732831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:20.732831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:00:20.732831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:20.732831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:00:20.732831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:20.732831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:20.732831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:20.732831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:20.732831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:20.732831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:20.732831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:20.732831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:20.732831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:20.732831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:20.732831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:20.732831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:20.732831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:20.733695 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:20.733695 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:20.733695 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:00:20.733695 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:00:20.733695 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:00:20.733695 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:00:20.733695 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:00:20.733695 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:00:20.733695 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 12 14:00:20.733695 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:20.733695 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 240/505] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 14:00:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:20.733695 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68b4af-7efe-4b21-9cd6-b3d61d9cc8fa None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:20.802696 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e41f68-646a-4cb9-afeb-f2c23e55b7b7 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:20.832522 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68b4af-7efe-4b21-9cd6-b3d61d9cc8fa None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:20.835958 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fbd635-c18d-40f3-b73c-a9086420da27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:20.836581 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fbd635-c18d-40f3-b73c-a9086420da27 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:20.837448 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fbd635-c18d-40f3-b73c-a9086420da27 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:20.837721 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fbd635-c18d-40f3-b73c-a9086420da27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:20.848052 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fbd635-c18d-40f3-b73c-a9086420da27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:20.849493 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fbd635-c18d-40f3-b73c-a9086420da27 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:20.852712 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:20.863407 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-12fbd635-c18d-40f3-b73c-a9086420da27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:20.863407 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:20.863407 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:20.863407 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:20.863407 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:20.863407 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:20.863407 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:20.863407 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:20.863407 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:20.863407 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:20.863407 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:20.863407 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:20.863407 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:20.863407 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:20.863407 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:20.863407 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:20.864402 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:20.864402 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:20.864402 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:20.864402 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:20.864402 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:20.864402 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:20.864402 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:20.864402 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:20.864402 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:20.864402 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:20.864402 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:20.864402 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:20.864402 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:20.864402 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:20.864402 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:20.864402 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:20.864402 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:20.865248 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:20.865248 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:20.865248 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:20.865248 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 266/506] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:20.865248 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fbd635-c18d-40f3-b73c-a9086420da27 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:20.870824 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d4e00c-d89d-4791-867b-08ff3a03734c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:20.871473 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4e00c-d89d-4791-867b-08ff3a03734c None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:20.871749 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4e00c-d89d-4791-867b-08ff3a03734c None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:20.872023 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4e00c-d89d-4791-867b-08ff3a03734c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:20.882463 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d4e00c-d89d-4791-867b-08ff3a03734c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:20.884384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d4e00c-d89d-4791-867b-08ff3a03734c None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:20.890193 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:20.898280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-b0d4e00c-d89d-4791-867b-08ff3a03734c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:20.898280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:20.898280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:20.898280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:20.898280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:20.898280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:20.898280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:20.898280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:20.898280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:20.898280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:20.898280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:20.898280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:20.898280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:20.898280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:20.898280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:20.898280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:20.899278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:20.899278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:20.899278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:20.899278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:20.899278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:20.899278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:20.899278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:20.899278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:20.899278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:20.899278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:20.899278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:20.899278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:20.899278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:20.899278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:20.899278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:20.899278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:20.899278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:20.900597 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:20.900597 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:20.900597 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:20.900597 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 241/507] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:20.901531 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d4e00c-d89d-4791-867b-08ff3a03734c None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:20.966762 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fbd635-c18d-40f3-b73c-a9086420da27 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:20.998978 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b6739c-1a5e-4762-b82c-eb1a107dd9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:20.999603 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b6739c-1a5e-4762-b82c-eb1a107dd9de None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:20.999885 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b6739c-1a5e-4762-b82c-eb1a107dd9de None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:20.999989 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d4e00c-d89d-4791-867b-08ff3a03734c None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:21.000159 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b6739c-1a5e-4762-b82c-eb1a107dd9de None admin] PATH_INFO: `/v3/roles/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:21.010873 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b6739c-1a5e-4762-b82c-eb1a107dd9de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:21.016106 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-66b6739c-1a5e-4762-b82c-eb1a107dd9de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:00:21.016106 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:21.016106 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:21.016106 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:21.016106 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:21.016106 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:21.016106 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:21.016106 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:21.016106 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:21.016106 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:21.016106 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:21.016106 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:21.016106 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:21.016106 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:21.016106 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:21.016106 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:21.017023 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:21.017023 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:21.017023 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:21.017023 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:21.017023 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:21.017023 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:21.017023 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:21.017023 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:21.017023 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:21.017023 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:21.017023 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:21.017023 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:21.017023 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:21.017023 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:21.017023 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:21.017023 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:21.017869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:21.017869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:21.017869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:21.017869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:21.017869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:21.017869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:21.017869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:21.017869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:21.017869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:21.017869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:21.017869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:00:21.017869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:21.017869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 267/508] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 12 14:00:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:21.018517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b6739c-1a5e-4762-b82c-eb1a107dd9de None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:21.035460 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1441ea7-f401-40b1-99a1-d70c72428811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:21.036066 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1441ea7-f401-40b1-99a1-d70c72428811 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:21.036345 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1441ea7-f401-40b1-99a1-d70c72428811 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:21.036638 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1441ea7-f401-40b1-99a1-d70c72428811 None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:21.037273 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1441ea7-f401-40b1-99a1-d70c72428811 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:21.037579 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1441ea7-f401-40b1-99a1-d70c72428811 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:21.038997 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1441ea7-f401-40b1-99a1-d70c72428811 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:21.051047 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 242/509] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:21.051831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1441ea7-f401-40b1-99a1-d70c72428811 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:21.085972 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:21.101241 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:21.120646 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b6739c-1a5e-4762-b82c-eb1a107dd9de None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:21.152954 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125f30c-8717-4b8a-a778-89d3673260b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:21.153484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125f30c-8717-4b8a-a778-89d3673260b6 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:21.153650 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1441ea7-f401-40b1-99a1-d70c72428811 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:21.153748 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125f30c-8717-4b8a-a778-89d3673260b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:21.154017 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125f30c-8717-4b8a-a778-89d3673260b6 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:21.154738 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2125f30c-8717-4b8a-a778-89d3673260b6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:21.155067 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2125f30c-8717-4b8a-a778-89d3673260b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:21.156485 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2125f30c-8717-4b8a-a778-89d3673260b6 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:21.187601 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85724b4-4b7c-4ad0-a5ab-1e272f06d693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:21.188200 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85724b4-4b7c-4ad0-a5ab-1e272f06d693 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:21.188475 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85724b4-4b7c-4ad0-a5ab-1e272f06d693 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:21.188744 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85724b4-4b7c-4ad0-a5ab-1e272f06d693 None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:21.189440 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85724b4-4b7c-4ad0-a5ab-1e272f06d693 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:21.189742 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85724b4-4b7c-4ad0-a5ab-1e272f06d693 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:21.191132 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85724b4-4b7c-4ad0-a5ab-1e272f06d693 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:21.191410 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 268/510] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Sep 12 14:00:21 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:21.192226 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125f30c-8717-4b8a-a778-89d3673260b6 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:21.206507 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 243/511] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:21.206507 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85724b4-4b7c-4ad0-a5ab-1e272f06d693 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:21.298749 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125f30c-8717-4b8a-a778-89d3673260b6 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:21.308703 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:21.318238 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85724b4-4b7c-4ad0-a5ab-1e272f06d693 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:21.333145 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf9123-92af-430a-9a88-f50201c317b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:21.333750 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf9123-92af-430a-9a88-f50201c317b1 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:21.334021 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf9123-92af-430a-9a88-f50201c317b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:21.334288 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf9123-92af-430a-9a88-f50201c317b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:21.334947 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbf9123-92af-430a-9a88-f50201c317b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:21.335244 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbf9123-92af-430a-9a88-f50201c317b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:21.336594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbf9123-92af-430a-9a88-f50201c317b1 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:21.342769 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:21.348194 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 269/512] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:21.349464 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf9123-92af-430a-9a88-f50201c317b1 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:21.352709 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d06536e-90ca-41a7-94b5-e98052929897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:21.353307 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d06536e-90ca-41a7-94b5-e98052929897 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:21.353583 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d06536e-90ca-41a7-94b5-e98052929897 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:21.353853 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d06536e-90ca-41a7-94b5-e98052929897 None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:21.354505 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d06536e-90ca-41a7-94b5-e98052929897 None admin] RBAC: Adding query filter params (name=service) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:21.354798 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d06536e-90ca-41a7-94b5-e98052929897 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:21.356180 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d06536e-90ca-41a7-94b5-e98052929897 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:21.364261 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 244/513] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 12 14:00:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:21.364964 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d06536e-90ca-41a7-94b5-e98052929897 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:21.449307 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf9123-92af-430a-9a88-f50201c317b1 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:21.462911 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d06536e-90ca-41a7-94b5-e98052929897 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:21.482553 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfbedd4-87e8-4b19-a509-dd74b97618c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:21.483191 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbedd4-87e8-4b19-a509-dd74b97618c9 None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:21.483470 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbedd4-87e8-4b19-a509-dd74b97618c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:21.483734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbedd4-87e8-4b19-a509-dd74b97618c9 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:21.484371 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfbedd4-87e8-4b19-a509-dd74b97618c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:21.485794 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfbedd4-87e8-4b19-a509-dd74b97618c9 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:21.496817 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024fd58c-04b6-4cb6-979c-adb535865eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:21.497408 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024fd58c-04b6-4cb6-979c-adb535865eaf None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:21.497697 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024fd58c-04b6-4cb6-979c-adb535865eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:21.497961 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024fd58c-04b6-4cb6-979c-adb535865eaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:21.507013 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 270/514] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Sep 12 14:00:21 2022] POST /identity/v3/users => generated 247 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:00:21.507820 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfbedd4-87e8-4b19-a509-dd74b97618c9 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:21.530681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:21.572348 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:21.586062 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024fd58c-04b6-4cb6-979c-adb535865eaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:21.589202 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024fd58c-04b6-4cb6-979c-adb535865eaf None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:21.627015 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfbedd4-87e8-4b19-a509-dd74b97618c9 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:21.669799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7bda0-1a96-41af-ae6e-42c5e68b2f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:21.670622 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7bda0-1a96-41af-ae6e-42c5e68b2f09 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:21.671026 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7bda0-1a96-41af-ae6e-42c5e68b2f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:21.671393 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7bda0-1a96-41af-ae6e-42c5e68b2f09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:21.678751 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7bda0-1a96-41af-ae6e-42c5e68b2f09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:21.681367 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7bda0-1a96-41af-ae6e-42c5e68b2f09 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:21.694305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-54e7bda0-1a96-41af-ae6e-42c5e68b2f09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:21.694305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:21.694305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:21.694305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:21.694305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:21.694305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:21.694305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:21.694305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:21.694305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:21.694305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:21.694305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:21.694305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:21.694305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:00:21.694305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:00:21.694305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:00:21.695976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:00:21.695976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:21.695976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:21.695976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:21.695976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:21.695976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:21.695976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:21.695976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:21.695976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:21.695976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:21.695976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:21.695976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:21.695976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:21.695976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:21.695976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:21.695976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:21.695976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:21.697108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:00:21.697108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:00:21.697108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:00:21.697108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:00:21.697108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:00:21.697108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:00:21.697108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:21.697108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:21.697108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 271/515] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:00:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:21.701094 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7bda0-1a96-41af-ae6e-42c5e68b2f09 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:21.710057 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 245/516] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:00:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 14:00:21.710807 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024fd58c-04b6-4cb6-979c-adb535865eaf None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:21.770174 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:21.816459 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024fd58c-04b6-4cb6-979c-adb535865eaf None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:21.816766 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7bda0-1a96-41af-ae6e-42c5e68b2f09 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:21.838101 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:21.858582 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f22a6a-1ce9-4422-8d96-f8f2280ad55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:21.859245 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f22a6a-1ce9-4422-8d96-f8f2280ad55e None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:21.859526 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f22a6a-1ce9-4422-8d96-f8f2280ad55e None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:21.859795 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f22a6a-1ce9-4422-8d96-f8f2280ad55e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:21.863338 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b02642-ccd6-4d31-879a-228d04ade65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:21.864125 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b02642-ccd6-4d31-879a-228d04ade65c None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:21.864545 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b02642-ccd6-4d31-879a-228d04ade65c None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:21.864956 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b02642-ccd6-4d31-879a-228d04ade65c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:21.869805 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f22a6a-1ce9-4422-8d96-f8f2280ad55e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:21.871303 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f22a6a-1ce9-4422-8d96-f8f2280ad55e None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:21.874624 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b02642-ccd6-4d31-879a-228d04ade65c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:21.876315 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b02642-ccd6-4d31-879a-228d04ade65c None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:21.886456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-f7f22a6a-1ce9-4422-8d96-f8f2280ad55e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:21.886456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:21.886456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:21.886456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:21.886456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:21.886456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:21.886456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:21.886456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:21.886456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:21.886456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:21.886456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:21.886456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:21.886456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:21.886456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:21.886456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:21.886456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:21.887586 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:21.887586 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:21.887586 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:21.887586 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:21.887586 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:21.887586 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:21.887586 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:21.887586 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:21.887586 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:21.887586 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:21.887586 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:21.887586 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:21.887586 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:21.887586 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:21.887586 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:21.887586 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:21.887586 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:21.892144 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:21.892144 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:21.892144 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:21.892144 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 246/517] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:21.892144 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f22a6a-1ce9-4422-8d96-f8f2280ad55e None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:21.895304 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-c7b02642-ccd6-4d31-879a-228d04ade65c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 14:00:21.895304 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:21.895304 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:21.895304 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:21.895304 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:21.895304 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:21.895304 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:21.895304 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:21.895304 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:21.895304 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:21.895304 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:21.895304 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:21.895304 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:00:21.895304 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:00:21.895304 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:00:21.895304 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:00:21.896195 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:21.896195 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:21.896195 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:00:21.896195 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:21.896195 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:00:21.896195 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:21.896195 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:21.896195 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:21.896195 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:21.896195 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:21.896195 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:21.896195 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:21.896195 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:21.896195 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:21.896195 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:21.896195 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:21.896195 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:21.896195 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:21.897339 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:21.897339 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:21.897339 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:00:21.897339 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:00:21.897339 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:00:21.897339 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:00:21.897339 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:00:21.897339 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:00:21.897339 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 14:00:21.897339 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:21.897339 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 272/518] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 14:00:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:21.898222 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b02642-ccd6-4d31-879a-228d04ade65c None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:22.002782 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f22a6a-1ce9-4422-8d96-f8f2280ad55e None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:22.007548 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b02642-ccd6-4d31-879a-228d04ade65c None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:22.009876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:22.038463 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1c411-30ad-4e37-88b0-747a25ed9ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:22.039350 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1c411-30ad-4e37-88b0-747a25ed9ba8 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:22.039632 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1c411-30ad-4e37-88b0-747a25ed9ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:22.040579 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1c411-30ad-4e37-88b0-747a25ed9ba8 None admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:22.041712 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f1c411-30ad-4e37-88b0-747a25ed9ba8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:22.042038 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f1c411-30ad-4e37-88b0-747a25ed9ba8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:22.043779 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f1c411-30ad-4e37-88b0-747a25ed9ba8 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:22.046182 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e6540-ddb4-42ce-bedc-c399e2adebb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:22.046806 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e6540-ddb4-42ce-bedc-c399e2adebb7 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:22.047111 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e6540-ddb4-42ce-bedc-c399e2adebb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:22.047380 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e6540-ddb4-42ce-bedc-c399e2adebb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:22.058131 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5e6540-ddb4-42ce-bedc-c399e2adebb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:22.059586 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5e6540-ddb4-42ce-bedc-c399e2adebb7 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:22.063296 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 247/519] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 12 14:00:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:22.064268 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1c411-30ad-4e37-88b0-747a25ed9ba8 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:22.074108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-2a5e6540-ddb4-42ce-bedc-c399e2adebb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:22.074108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:22.074108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:22.074108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:22.074108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:22.074108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:22.074108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:22.074108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:22.074108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:22.074108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:22.074108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:22.074108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:22.074108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:22.074108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:22.074108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:22.074108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:22.075246 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:22.075246 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:22.075246 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:22.075246 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:22.075246 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:22.075246 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:22.075246 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:22.075246 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:22.075246 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:22.075246 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:22.075246 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:22.075246 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:22.075246 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:22.075246 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:22.075246 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:22.075246 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:22.075246 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:22.076136 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:22.076136 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:22.076136 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:22.079111 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 273/520] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:22.079803 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e6540-ddb4-42ce-bedc-c399e2adebb7 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:22.107738 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:22.176805 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1c411-30ad-4e37-88b0-747a25ed9ba8 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:22.185347 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e6540-ddb4-42ce-bedc-c399e2adebb7 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:22.212701 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d0d60b-5621-490a-904d-573150dc841c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:22.213463 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0d60b-5621-490a-904d-573150dc841c None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:22.214062 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0d60b-5621-490a-904d-573150dc841c None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:22.214344 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0d60b-5621-490a-904d-573150dc841c None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:22.215221 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d0d60b-5621-490a-904d-573150dc841c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:22.215525 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d0d60b-5621-490a-904d-573150dc841c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:22.216985 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d0d60b-5621-490a-904d-573150dc841c None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:22.221294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8633cfe-f65d-49af-88d1-f838f877a9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:22.221897 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8633cfe-f65d-49af-88d1-f838f877a9a7 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:22.222173 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8633cfe-f65d-49af-88d1-f838f877a9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:22.222440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8633cfe-f65d-49af-88d1-f838f877a9a7 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:22.223200 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8633cfe-f65d-49af-88d1-f838f877a9a7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:22.223508 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8633cfe-f65d-49af-88d1-f838f877a9a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:22.224899 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8633cfe-f65d-49af-88d1-f838f877a9a7 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:22.232800 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 248/521] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:22.232800 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d0d60b-5621-490a-904d-573150dc841c None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:22.242799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 274/522] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Sep 12 14:00:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:22.244804 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8633cfe-f65d-49af-88d1-f838f877a9a7 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:22.264774 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:22.341775 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:22.347348 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d0d60b-5621-490a-904d-573150dc841c None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:22.360435 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8633cfe-f65d-49af-88d1-f838f877a9a7 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:22.383373 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150fecfa-5e0c-4ac4-9cc9-e21dce43f0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:22.384849 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150fecfa-5e0c-4ac4-9cc9-e21dce43f0a9 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:22.385283 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150fecfa-5e0c-4ac4-9cc9-e21dce43f0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:22.385558 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150fecfa-5e0c-4ac4-9cc9-e21dce43f0a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:22.386281 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150fecfa-5e0c-4ac4-9cc9-e21dce43f0a9 None admin] RBAC: Adding query filter params (role.id=1c413f4e78ef45b7a821bb884f9cea07, scope.project.id=f666a617f8d14eb18bbeb076032b82f2, user.id=854ab2dcec5a48b18f409b33cb61b5da) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:22.386588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150fecfa-5e0c-4ac4-9cc9-e21dce43f0a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:22.388034 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150fecfa-5e0c-4ac4-9cc9-e21dce43f0a9 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:22.402872 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 249/523] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 14:00:22 2022] GET /identity/v3/role_assignments?user.id=854ab2dcec5a48b18f409b33cb61b5da&scope.project.id=f666a617f8d14eb18bbeb076032b82f2&role.id=1c413f4e78ef45b7a821bb884f9cea07 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:22.402872 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150fecfa-5e0c-4ac4-9cc9-e21dce43f0a9 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:22.405532 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96254520-3d25-4de3-828a-cdbccc3677ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:22.406382 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96254520-3d25-4de3-828a-cdbccc3677ef None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:22.406753 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96254520-3d25-4de3-828a-cdbccc3677ef None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:22.407190 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96254520-3d25-4de3-828a-cdbccc3677ef None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:22.407930 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96254520-3d25-4de3-828a-cdbccc3677ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:22.408345 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96254520-3d25-4de3-828a-cdbccc3677ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:22.409862 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96254520-3d25-4de3-828a-cdbccc3677ef None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:22.427078 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 275/524] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:22.428352 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96254520-3d25-4de3-828a-cdbccc3677ef None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:22.520459 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:22.533215 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150fecfa-5e0c-4ac4-9cc9-e21dce43f0a9 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:22.572523 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96254520-3d25-4de3-828a-cdbccc3677ef None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:22.573746 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51f5ed3-5b12-44a5-b687-afd1b33f7dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:22.574648 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f5ed3-5b12-44a5-b687-afd1b33f7dbe None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:22.575207 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f5ed3-5b12-44a5-b687-afd1b33f7dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:22.575613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f5ed3-5b12-44a5-b687-afd1b33f7dbe None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:22.585685 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51f5ed3-5b12-44a5-b687-afd1b33f7dbe None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:22.587481 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51f5ed3-5b12-44a5-b687-afd1b33f7dbe None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:22.603234 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-a51f5ed3-5b12-44a5-b687-afd1b33f7dbe None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 12 14:00:22.603234 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:22.603234 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:22.603234 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:22.603234 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:22.603234 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:22.603234 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:22.603234 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:22.603234 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:22.603234 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:22.603234 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:22.603234 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:22.603234 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:00:22.603234 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:00:22.603234 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:00:22.603234 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:00:22.604354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:22.604354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:22.604354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:00:22.604354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:22.604354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:00:22.604354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:22.604354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:22.604354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:22.604354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:22.604354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:22.604354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:22.604354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:22.604354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:22.604354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:22.604354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:22.604354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:22.604354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:22.604354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:22.605541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:22.605541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:22.605541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:00:22.605541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:00:22.605541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:00:22.605541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:00:22.605541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:00:22.605541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:00:22.605541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 12 14:00:22.605541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:22.605541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 250/525] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 12 14:00:22 2022] GET /identity/v3/users/neutron => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:22.605541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51f5ed3-5b12-44a5-b687-afd1b33f7dbe None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:22.616989 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc29f715-7b9f-4b58-92bd-84cbcdb16462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:22.617682 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc29f715-7b9f-4b58-92bd-84cbcdb16462 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:22.618599 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc29f715-7b9f-4b58-92bd-84cbcdb16462 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:22.618950 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc29f715-7b9f-4b58-92bd-84cbcdb16462 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:22.635946 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:22.693452 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc29f715-7b9f-4b58-92bd-84cbcdb16462 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:22.695052 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc29f715-7b9f-4b58-92bd-84cbcdb16462 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:22.748098 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51f5ed3-5b12-44a5-b687-afd1b33f7dbe None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:22.794556 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f0868d-1f15-4edd-9f60-11eb37ff2f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:22.795145 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f0868d-1f15-4edd-9f60-11eb37ff2f77 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:22.795419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f0868d-1f15-4edd-9f60-11eb37ff2f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:22.795687 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f0868d-1f15-4edd-9f60-11eb37ff2f77 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:22.805145 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f0868d-1f15-4edd-9f60-11eb37ff2f77 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:22.807782 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f0868d-1f15-4edd-9f60-11eb37ff2f77 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:22.810777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 276/526] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:00:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 14:00:22.811894 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc29f715-7b9f-4b58-92bd-84cbcdb16462 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:22.825854 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-87f0868d-1f15-4edd-9f60-11eb37ff2f77 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:22.825854 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:22.825854 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:22.825854 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:22.825854 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:22.825854 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:22.825854 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:22.825854 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:22.825854 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:22.825854 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:22.825854 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:22.825854 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:22.825854 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:00:22.825854 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:00:22.825854 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:00:22.826763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:00:22.826763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:22.826763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:22.826763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:22.826763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:22.826763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:22.826763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:22.826763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:22.826763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:22.826763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:22.826763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:22.826763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:22.826763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:22.826763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:22.826763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:22.826763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:22.826763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:22.827754 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:00:22.827754 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:00:22.827754 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:00:22.827754 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:00:22.827754 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:00:22.827754 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:00:22.827754 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:22.827754 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:22.827754 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 251/527] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:00:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:22.828215 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f0868d-1f15-4edd-9f60-11eb37ff2f77 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:22.845500 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:22.875427 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:22.949892 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc29f715-7b9f-4b58-92bd-84cbcdb16462 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:22.967476 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f0868d-1f15-4edd-9f60-11eb37ff2f77 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:22.995980 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda5877-2217-41cf-a2b5-ecf878734bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:22.996594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda5877-2217-41cf-a2b5-ecf878734bcc None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:22.996874 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda5877-2217-41cf-a2b5-ecf878734bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:22.997148 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda5877-2217-41cf-a2b5-ecf878734bcc None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:22.998153 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda5877-2217-41cf-a2b5-ecf878734bcc None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:22.998516 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda5877-2217-41cf-a2b5-ecf878734bcc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:23.000659 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda5877-2217-41cf-a2b5-ecf878734bcc None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:23.011143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ea3e7-b7e3-49b1-982a-8dde0d132cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:23.012935 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ea3e7-b7e3-49b1-982a-8dde0d132cda None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:23.013421 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ea3e7-b7e3-49b1-982a-8dde0d132cda None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:23.013891 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ea3e7-b7e3-49b1-982a-8dde0d132cda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:23.020616 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 277/528] 10.222.0.42 () {56 vars in 1139 bytes} [Mon Sep 12 14:00:22 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:23.023696 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda5877-2217-41cf-a2b5-ecf878734bcc None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:23.030948 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650ea3e7-b7e3-49b1-982a-8dde0d132cda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:23.032637 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650ea3e7-b7e3-49b1-982a-8dde0d132cda None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:23.053217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-650ea3e7-b7e3-49b1-982a-8dde0d132cda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:23.053217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:23.053217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:23.053217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:23.053217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:23.053217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:23.053217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:23.053217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:23.053217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:23.053217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:23.053217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:23.053217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:23.053217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:23.053217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:23.053217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:23.053217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:23.054165 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:23.054165 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:23.054165 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:23.054165 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:23.054165 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:23.054165 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:23.054165 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:23.054165 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:23.054165 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:23.054165 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:23.054165 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:23.054165 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:23.054165 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:23.054165 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:23.054165 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:23.054165 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:23.054165 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:23.056118 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:23.056118 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:23.056118 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:23.061583 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 252/529] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:23.061583 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ea3e7-b7e3-49b1-982a-8dde0d132cda None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:23.125532 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:23.142063 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda5877-2217-41cf-a2b5-ecf878734bcc None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:23.143940 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:23.168676 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ea3e7-b7e3-49b1-982a-8dde0d132cda None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:23.182467 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c194c-d2c0-471b-9e78-da7185195463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:23.183437 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c194c-d2c0-471b-9e78-da7185195463 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:23.183922 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c194c-d2c0-471b-9e78-da7185195463 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:23.184469 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c194c-d2c0-471b-9e78-da7185195463 None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:23.185363 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c194c-d2c0-471b-9e78-da7185195463 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:23.186032 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c194c-d2c0-471b-9e78-da7185195463 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:23.188093 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c194c-d2c0-471b-9e78-da7185195463 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:23.204250 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08430ef-5ed0-4e79-8952-2e493d955d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:23.204861 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08430ef-5ed0-4e79-8952-2e493d955d31 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:23.205140 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08430ef-5ed0-4e79-8952-2e493d955d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:23.205414 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08430ef-5ed0-4e79-8952-2e493d955d31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:23.210641 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 278/530] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 12 14:00:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:23.211770 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c194c-d2c0-471b-9e78-da7185195463 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:23.282186 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08430ef-5ed0-4e79-8952-2e493d955d31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:23.283610 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08430ef-5ed0-4e79-8952-2e493d955d31 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:23.332947 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c194c-d2c0-471b-9e78-da7185195463 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:23.349814 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:23.379734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df287b-cab7-445d-b905-795de37d1a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:23.380560 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df287b-cab7-445d-b905-795de37d1a30 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:23.380933 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df287b-cab7-445d-b905-795de37d1a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:23.381300 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df287b-cab7-445d-b905-795de37d1a30 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:23.382574 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df287b-cab7-445d-b905-795de37d1a30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:23.383015 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df287b-cab7-445d-b905-795de37d1a30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:23.384997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df287b-cab7-445d-b905-795de37d1a30 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:23.400591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 253/531] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:00:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 14:00:23.401755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08430ef-5ed0-4e79-8952-2e493d955d31 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:23.403565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 279/532] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:23.404438 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df287b-cab7-445d-b905-795de37d1a30 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:23.435707 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:23.523917 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df287b-cab7-445d-b905-795de37d1a30 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:23.533804 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08430ef-5ed0-4e79-8952-2e493d955d31 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:23.565851 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c1039-f370-46b7-ae06-0089bd10d477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:23.567789 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c1039-f370-46b7-ae06-0089bd10d477 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:23.568244 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c1039-f370-46b7-ae06-0089bd10d477 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:23.568656 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c1039-f370-46b7-ae06-0089bd10d477 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:23.578270 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dacdde-bdea-42e8-b412-fd4803ee6b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:23.579127 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dacdde-bdea-42e8-b412-fd4803ee6b66 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:23.579504 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dacdde-bdea-42e8-b412-fd4803ee6b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:23.579877 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dacdde-bdea-42e8-b412-fd4803ee6b66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:23.580705 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dacdde-bdea-42e8-b412-fd4803ee6b66 None admin] RBAC: Adding query filter params (role.id=1c413f4e78ef45b7a821bb884f9cea07, scope.project.id=f666a617f8d14eb18bbeb076032b82f2, user.id=2092a9639067461c9a837a891f56889b) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:23.581103 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dacdde-bdea-42e8-b412-fd4803ee6b66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:23.581697 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c1039-f370-46b7-ae06-0089bd10d477 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:23.582723 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dacdde-bdea-42e8-b412-fd4803ee6b66 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:23.583516 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c1039-f370-46b7-ae06-0089bd10d477 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:23.595509 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 254/533] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 14:00:23 2022] GET /identity/v3/role_assignments?user.id=2092a9639067461c9a837a891f56889b&scope.project.id=f666a617f8d14eb18bbeb076032b82f2&role.id=1c413f4e78ef45b7a821bb884f9cea07 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:23.596627 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dacdde-bdea-42e8-b412-fd4803ee6b66 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:23.601281 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-326c1039-f370-46b7-ae06-0089bd10d477 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:23.601281 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:23.601281 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:23.601281 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:23.601281 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:23.601281 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:23.601281 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:23.601281 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:23.601281 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:23.601281 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:23.601281 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:23.601281 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:23.601281 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:23.601281 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:23.601281 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:23.601281 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:23.602316 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:23.602316 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:23.602316 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:23.602316 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:23.602316 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:23.602316 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:23.602316 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:23.602316 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:23.602316 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:23.602316 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:23.602316 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:23.602316 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:23.602316 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:23.602316 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:23.602316 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:23.602316 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:23.602316 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:23.604019 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:23.604019 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:23.604019 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:23.606486 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 280/534] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:23.618435 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c1039-f370-46b7-ae06-0089bd10d477 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:23.675610 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:23.754246 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:23.793342 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c1039-f370-46b7-ae06-0089bd10d477 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:23.855399 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f224c-5185-4b58-9545-2c5d0bf2bcb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:23.856126 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f224c-5185-4b58-9545-2c5d0bf2bcb6 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:23.856475 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f224c-5185-4b58-9545-2c5d0bf2bcb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:23.856813 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f224c-5185-4b58-9545-2c5d0bf2bcb6 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:23.857535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71f224c-5185-4b58-9545-2c5d0bf2bcb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:23.857910 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71f224c-5185-4b58-9545-2c5d0bf2bcb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:23.858607 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dacdde-bdea-42e8-b412-fd4803ee6b66 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:23.859420 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71f224c-5185-4b58-9545-2c5d0bf2bcb6 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:23.874726 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 281/535] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:23.876153 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8c4fb-55ed-4455-bfe1-23faab931868 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:23.876506 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8c4fb-55ed-4455-bfe1-23faab931868 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:23.876843 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8c4fb-55ed-4455-bfe1-23faab931868 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:23.879118 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 282/536] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:23.882798 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8c4fb-55ed-4455-bfe1-23faab931868 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:23.902809 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57f5a2c-6c4e-4d76-94a9-d6d25be2f065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:23.903721 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f5a2c-6c4e-4d76-94a9-d6d25be2f065 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:23.904139 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f5a2c-6c4e-4d76-94a9-d6d25be2f065 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:23.904541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f5a2c-6c4e-4d76-94a9-d6d25be2f065 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:23.914308 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57f5a2c-6c4e-4d76-94a9-d6d25be2f065 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:23.916945 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57f5a2c-6c4e-4d76-94a9-d6d25be2f065 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:23.939800 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-f57f5a2c-6c4e-4d76-94a9-d6d25be2f065 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:23.939800 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:23.939800 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:23.939800 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:23.939800 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:23.939800 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:23.939800 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:23.939800 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:23.939800 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:23.939800 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:23.939800 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:23.939800 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:23.939800 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:00:23.939800 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:00:23.939800 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:00:23.940670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:00:23.940670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:23.940670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:23.940670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:23.940670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:23.940670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:23.940670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:23.940670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:23.940670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:23.940670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:23.940670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:23.940670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:23.940670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:23.940670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:23.940670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:23.940670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:23.940670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:23.941501 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:00:23.941501 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:00:23.941501 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:00:23.941501 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:00:23.941501 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:00:23.941501 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:00:23.941501 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:23.941501 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:23.944069 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 255/537] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:00:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:23.945412 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc82a24-a510-4a25-9c1a-0f3ccc104a82 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:23.946885 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc82a24-a510-4a25-9c1a-0f3ccc104a82 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:23.947385 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc82a24-a510-4a25-9c1a-0f3ccc104a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:23.977373 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-2fc82a24-a510-4a25-9c1a-0f3ccc104a82 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:23.981036 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-2fc82a24-a510-4a25-9c1a-0f3ccc104a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:24.035723 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8c4fb-55ed-4455-bfe1-23faab931868 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:24.083608 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54380cf1-ba28-4d16-9c63-d149738dbd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:24.084330 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54380cf1-ba28-4d16-9c63-d149738dbd96 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:24.084557 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54380cf1-ba28-4d16-9c63-d149738dbd96 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:24.084827 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54380cf1-ba28-4d16-9c63-d149738dbd96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:24.090233 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:24.098740 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54380cf1-ba28-4d16-9c63-d149738dbd96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:24.100481 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54380cf1-ba28-4d16-9c63-d149738dbd96 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:24.115205 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-54380cf1-ba28-4d16-9c63-d149738dbd96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:24.115205 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:24.115205 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:24.115205 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:24.115205 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:24.115205 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:24.115205 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:24.115205 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:24.115205 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:24.115205 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:24.115205 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:24.115205 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:24.115205 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:00:24.115205 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:00:24.115205 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:00:24.116088 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:00:24.116088 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:24.116088 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:24.116088 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:24.116088 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:24.116088 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:24.116088 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:24.116088 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:24.116088 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:24.116088 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:24.116088 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:24.116088 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:24.116088 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:24.116088 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:24.116088 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:24.116088 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:24.116088 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:24.116931 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:00:24.116931 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:00:24.116931 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:00:24.116931 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:00:24.116931 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:00:24.116931 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:00:24.116931 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:24.116931 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:24.116931 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 283/538] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:00:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:24.118440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54380cf1-ba28-4d16-9c63-d149738dbd96 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:24.132851 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 256/539] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:24.134737 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc82a24-a510-4a25-9c1a-0f3ccc104a82 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:24.213988 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:24.231050 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54380cf1-ba28-4d16-9c63-d149738dbd96 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:24.270313 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5927f5b-1546-4a12-94c5-640e910285f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:24.271217 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5927f5b-1546-4a12-94c5-640e910285f8 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:24.271641 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5927f5b-1546-4a12-94c5-640e910285f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:24.272120 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5927f5b-1546-4a12-94c5-640e910285f8 None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:24.273011 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5927f5b-1546-4a12-94c5-640e910285f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:24.273547 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5927f5b-1546-4a12-94c5-640e910285f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:24.275314 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5927f5b-1546-4a12-94c5-640e910285f8 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:24.289721 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 284/540] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 12 14:00:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:24.299111 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5927f5b-1546-4a12-94c5-640e910285f8 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:24.413343 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc82a24-a510-4a25-9c1a-0f3ccc104a82 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:24.447193 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:24.476800 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726465eb-4c24-415a-8019-784353047fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:24.477983 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726465eb-4c24-415a-8019-784353047fbd None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:24.478256 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726465eb-4c24-415a-8019-784353047fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:24.478514 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726465eb-4c24-415a-8019-784353047fbd None admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:24.479243 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726465eb-4c24-415a-8019-784353047fbd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:24.479537 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726465eb-4c24-415a-8019-784353047fbd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:24.480975 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726465eb-4c24-415a-8019-784353047fbd None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:24.494012 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 257/541] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 12 14:00:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:24.504335 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726465eb-4c24-415a-8019-784353047fbd None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:24.515065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5927f5b-1546-4a12-94c5-640e910285f8 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:24.575773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8414d-7b61-45e9-94a8-0302b927bd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:24.576392 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8414d-7b61-45e9-94a8-0302b927bd39 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:24.576667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8414d-7b61-45e9-94a8-0302b927bd39 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:24.576932 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8414d-7b61-45e9-94a8-0302b927bd39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:24.577668 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd8414d-7b61-45e9-94a8-0302b927bd39 None admin] RBAC: Adding query filter params (role.id=1c413f4e78ef45b7a821bb884f9cea07, scope.project.id=f666a617f8d14eb18bbeb076032b82f2, user.id=8da38bc8dd2f4cf798649ce96097a4b8) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:24.577972 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd8414d-7b61-45e9-94a8-0302b927bd39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:24.580076 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd8414d-7b61-45e9-94a8-0302b927bd39 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:24.589069 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 285/542] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 14:00:24 2022] GET /identity/v3/role_assignments?user.id=8da38bc8dd2f4cf798649ce96097a4b8&scope.project.id=f666a617f8d14eb18bbeb076032b82f2&role.id=1c413f4e78ef45b7a821bb884f9cea07 => generated 259 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:24.591144 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8414d-7b61-45e9-94a8-0302b927bd39 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:24.655257 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:24.696006 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726465eb-4c24-415a-8019-784353047fbd None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:24.733634 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85640013-5fdf-407d-9efd-dc6e40265b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:24.734265 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85640013-5fdf-407d-9efd-dc6e40265b12 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:24.734547 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85640013-5fdf-407d-9efd-dc6e40265b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:24.734987 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85640013-5fdf-407d-9efd-dc6e40265b12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:24.736023 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85640013-5fdf-407d-9efd-dc6e40265b12 None admin] RBAC: Adding query filter params (role.id=1c413f4e78ef45b7a821bb884f9cea07, scope.project.id=f666a617f8d14eb18bbeb076032b82f2, user.id=865b1647468c4efe8efe4febe2203a58) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:24.736343 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85640013-5fdf-407d-9efd-dc6e40265b12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:24.738072 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85640013-5fdf-407d-9efd-dc6e40265b12 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:24.747528 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 258/543] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 14:00:24 2022] GET /identity/v3/role_assignments?user.id=865b1647468c4efe8efe4febe2203a58&scope.project.id=f666a617f8d14eb18bbeb076032b82f2&role.id=1c413f4e78ef45b7a821bb884f9cea07 => generated 259 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:24.748886 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc521c53-f908-493d-89bb-66d687651f36 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:24.749172 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc521c53-f908-493d-89bb-66d687651f36 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:24.749444 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc521c53-f908-493d-89bb-66d687651f36 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:24.755670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 259/544] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:24.757295 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8f762-b266-47a0-b663-183634582191 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:24.757589 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8f762-b266-47a0-b663-183634582191 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:24.757868 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8f762-b266-47a0-b663-183634582191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:24.769302 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8414d-7b61-45e9-94a8-0302b927bd39 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:24.789152 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-97c8f762-b266-47a0-b663-183634582191 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:24.790314 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-97c8f762-b266-47a0-b663-183634582191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:24.810437 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:24.843664 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5a685-6d9a-44ac-b8e6-57ea343b1f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:24.845241 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5a685-6d9a-44ac-b8e6-57ea343b1f48 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:24.845241 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5a685-6d9a-44ac-b8e6-57ea343b1f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:24.845241 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5a685-6d9a-44ac-b8e6-57ea343b1f48 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:24.887618 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb5a685-6d9a-44ac-b8e6-57ea343b1f48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:24.891960 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 260/545] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:24.897386 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-9fb5a685-6d9a-44ac-b8e6-57ea343b1f48 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:00:24.897386 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:24.897386 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:24.897386 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:24.897386 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:24.897386 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:24.897386 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:24.897386 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:24.897386 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:24.897386 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:24.897386 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:24.897386 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:24.897386 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:24.897386 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:24.897386 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:24.897386 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:24.898339 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:24.898339 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:24.898339 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:24.898339 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:24.898339 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:24.898339 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:24.898339 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:24.898339 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:24.898339 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:24.898339 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:24.898339 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:24.898339 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:24.898339 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:24.898339 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:24.898339 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:24.898339 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:24.899199 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:24.899199 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:24.899199 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:24.899199 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:24.899199 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:24.899199 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:24.899199 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:24.899199 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:24.899199 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:24.899199 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:24.899199 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:00:24.899199 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:24.899199 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 286/546] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 12 14:00:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:24.906539 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c8f762-b266-47a0-b663-183634582191 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:24.995027 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:25.049793 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c8f762-b266-47a0-b663-183634582191 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:25.098245 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78498358-69de-4148-bffa-f589048f693a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:25.098863 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78498358-69de-4148-bffa-f589048f693a None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:25.099151 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78498358-69de-4148-bffa-f589048f693a None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:25.099415 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78498358-69de-4148-bffa-f589048f693a None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:25.100073 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78498358-69de-4148-bffa-f589048f693a None admin] RBAC: Adding query filter params (name=service) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:25.100368 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78498358-69de-4148-bffa-f589048f693a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:25.101824 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78498358-69de-4148-bffa-f589048f693a None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:25.114170 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 261/547] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 12 14:00:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:25.120355 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5a685-6d9a-44ac-b8e6-57ea343b1f48 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:25.263897 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5a685-6d9a-44ac-b8e6-57ea343b1f48 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:25.297373 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78498358-69de-4148-bffa-f589048f693a None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:25.329420 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded37fa-9d6e-4b8b-b68e-6436bf9e4a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:25.330411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded37fa-9d6e-4b8b-b68e-6436bf9e4a6b None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:25.330686 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded37fa-9d6e-4b8b-b68e-6436bf9e4a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:25.330997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded37fa-9d6e-4b8b-b68e-6436bf9e4a6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:25.361841 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded37fa-9d6e-4b8b-b68e-6436bf9e4a6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:25.363312 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded37fa-9d6e-4b8b-b68e-6436bf9e4a6b None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:25.386156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-9ded37fa-9d6e-4b8b-b68e-6436bf9e4a6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:25.386156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:25.386156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:25.386156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:25.386156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:25.386156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:25.386156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:25.386156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:25.386156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:25.386156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:25.386156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:25.386156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:25.386156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:25.386156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:25.386156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:25.386156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:25.387109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:25.387109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:25.387109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:25.387109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:25.387109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:25.387109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:25.387109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:25.387109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:25.387109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:25.387109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:25.387109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:25.387109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:25.387109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:25.387109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:25.387109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:25.387109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:25.387109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:25.387945 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:25.387945 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:25.387945 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:25.389838 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 287/548] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:25.396218 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded37fa-9d6e-4b8b-b68e-6436bf9e4a6b None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:25.439597 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:25.492983 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78498358-69de-4148-bffa-f589048f693a None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:25.557388 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86f6b4-7b25-428c-849b-1930175589a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:25.557947 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86f6b4-7b25-428c-849b-1930175589a2 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:25.558232 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86f6b4-7b25-428c-849b-1930175589a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:25.558512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86f6b4-7b25-428c-849b-1930175589a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:25.563893 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:25.569672 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b86f6b4-7b25-428c-849b-1930175589a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:25.575989 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-8b86f6b4-7b25-428c-849b-1930175589a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:00:25.575989 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:25.575989 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:25.575989 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:25.575989 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:25.575989 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:25.575989 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:25.575989 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:25.575989 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:25.575989 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:25.575989 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:25.575989 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:25.575989 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:25.575989 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:25.575989 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:25.575989 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:25.577389 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:25.577389 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:25.577389 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:25.577389 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:25.577389 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:25.577389 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:25.577389 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:25.577389 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:25.577389 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:25.577389 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:25.577389 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:25.577389 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:25.577389 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:25.577389 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:25.577389 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:25.577389 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:25.578339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:25.578339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:25.578339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:25.578339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:25.578339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:25.578339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:25.578339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:25.578339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:25.578339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:25.578339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:25.578339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:00:25.578339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:25.578339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 262/549] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 12 14:00:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:00:25.586322 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86f6b4-7b25-428c-849b-1930175589a2 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:25.598449 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded37fa-9d6e-4b8b-b68e-6436bf9e4a6b None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:25.665843 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7fd6f8-6f9b-4f81-a115-fd7235c38d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:25.666461 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7fd6f8-6f9b-4f81-a115-fd7235c38d73 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:25.666737 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7fd6f8-6f9b-4f81-a115-fd7235c38d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:25.667029 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7fd6f8-6f9b-4f81-a115-fd7235c38d73 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:25.667687 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7fd6f8-6f9b-4f81-a115-fd7235c38d73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:25.667984 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7fd6f8-6f9b-4f81-a115-fd7235c38d73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:25.669372 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7fd6f8-6f9b-4f81-a115-fd7235c38d73 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:25.682544 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 288/550] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:25.692091 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7fd6f8-6f9b-4f81-a115-fd7235c38d73 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:25.770367 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86f6b4-7b25-428c-849b-1930175589a2 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:25.837718 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd675918-d536-4037-8ad4-101b27e3cf5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:25.837718 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd675918-d536-4037-8ad4-101b27e3cf5c None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:25.837718 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd675918-d536-4037-8ad4-101b27e3cf5c None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:25.837718 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd675918-d536-4037-8ad4-101b27e3cf5c None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:25.838110 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd675918-d536-4037-8ad4-101b27e3cf5c None admin] RBAC: Adding query filter params (name=service) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:25.838110 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd675918-d536-4037-8ad4-101b27e3cf5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:25.839624 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd675918-d536-4037-8ad4-101b27e3cf5c None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:25.851242 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 263/551] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 12 14:00:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:25.856597 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:25.857330 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd675918-d536-4037-8ad4-101b27e3cf5c None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:25.872550 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7fd6f8-6f9b-4f81-a115-fd7235c38d73 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:25.943556 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e575b-6920-4277-bf26-c0e1140433e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:25.944206 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e575b-6920-4277-bf26-c0e1140433e4 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:25.944481 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e575b-6920-4277-bf26-c0e1140433e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:25.944748 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e575b-6920-4277-bf26-c0e1140433e4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:25.962366 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430e575b-6920-4277-bf26-c0e1140433e4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:25.963822 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430e575b-6920-4277-bf26-c0e1140433e4 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:25.967687 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:25.983484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-430e575b-6920-4277-bf26-c0e1140433e4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 12 14:00:25.983484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:25.983484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:25.983484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:25.983484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:25.983484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:25.983484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:25.983484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:25.983484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:25.983484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:25.983484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:25.983484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:25.983484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:00:25.983484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:00:25.983484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:00:25.983484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:00:25.984531 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:25.984531 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:25.984531 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:00:25.984531 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:25.984531 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:00:25.984531 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:25.984531 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:25.984531 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:25.984531 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:25.984531 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:25.984531 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:25.984531 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:25.984531 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:25.984531 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:25.984531 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:25.984531 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:25.984531 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:25.984531 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:25.985936 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:25.985936 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:25.985936 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:00:25.985936 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:00:25.985936 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:00:25.985936 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:00:25.985936 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:00:25.985936 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:00:25.985936 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 12 14:00:25.985936 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:25.985936 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 289/552] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 12 14:00:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:00:25.992076 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e575b-6920-4277-bf26-c0e1140433e4 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:26.034049 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd675918-d536-4037-8ad4-101b27e3cf5c None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:26.070398 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d7aa53-ccd0-42e0-b6f8-d239d957d995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:26.071437 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7aa53-ccd0-42e0-b6f8-d239d957d995 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:26.072198 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7aa53-ccd0-42e0-b6f8-d239d957d995 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:26.072605 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7aa53-ccd0-42e0-b6f8-d239d957d995 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:26.084450 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d7aa53-ccd0-42e0-b6f8-d239d957d995 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:26.086161 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d7aa53-ccd0-42e0-b6f8-d239d957d995 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:26.101647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-c1d7aa53-ccd0-42e0-b6f8-d239d957d995 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:26.101647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:26.101647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:26.101647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:26.101647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:26.101647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:26.101647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:26.101647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:26.101647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:26.101647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:26.101647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:26.101647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:26.101647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:26.101647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:26.101647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:26.101647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:26.103370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:26.103370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:26.103370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:26.103370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:26.103370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:26.103370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:26.103370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:26.103370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:26.103370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:26.103370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:26.103370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:26.103370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:26.103370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:26.103370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:26.103370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:26.103370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:26.103370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:26.104377 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:26.104377 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:26.104377 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:26.106125 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 264/553] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:00:26.107798 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e4f8a-b944-47db-9ffa-421d568f6936 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:26.108216 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e4f8a-b944-47db-9ffa-421d568f6936 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:26.108639 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e4f8a-b944-47db-9ffa-421d568f6936 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:26.110679 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 265/554] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:26.111625 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e4f8a-b944-47db-9ffa-421d568f6936 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:26.116954 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e575b-6920-4277-bf26-c0e1140433e4 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:26.151650 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9273337-2921-4936-8996-cabc08790743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:26.151650 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9273337-2921-4936-8996-cabc08790743 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:26.151951 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9273337-2921-4936-8996-cabc08790743 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:26.152179 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9273337-2921-4936-8996-cabc08790743 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:26.152904 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9273337-2921-4936-8996-cabc08790743 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:26.153207 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9273337-2921-4936-8996-cabc08790743 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:26.154616 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9273337-2921-4936-8996-cabc08790743 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:26.172438 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 290/555] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Sep 12 14:00:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:26.173660 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a23aa-ce9d-491c-8407-84c525a9c3fe None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:26.173943 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a23aa-ce9d-491c-8407-84c525a9c3fe None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:26.174210 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a23aa-ce9d-491c-8407-84c525a9c3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:26.192723 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-f14a23aa-ce9d-491c-8407-84c525a9c3fe None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:26.193906 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f14a23aa-ce9d-491c-8407-84c525a9c3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:26.195921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:26.230705 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e4f8a-b944-47db-9ffa-421d568f6936 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:26.239817 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:26.271154 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 291/556] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:26.272230 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a23aa-ce9d-491c-8407-84c525a9c3fe None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:26.275035 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669bfc86-9eba-47af-9723-ed7d479d8991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:26.275844 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bfc86-9eba-47af-9723-ed7d479d8991 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:26.277073 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bfc86-9eba-47af-9723-ed7d479d8991 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:26.277494 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bfc86-9eba-47af-9723-ed7d479d8991 None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:26.278270 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669bfc86-9eba-47af-9723-ed7d479d8991 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:26.278706 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669bfc86-9eba-47af-9723-ed7d479d8991 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:26.280304 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669bfc86-9eba-47af-9723-ed7d479d8991 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:26.294416 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 266/557] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:26.302330 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669bfc86-9eba-47af-9723-ed7d479d8991 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:26.433351 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669bfc86-9eba-47af-9723-ed7d479d8991 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:26.473103 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f44b79-6a0d-43d9-af5d-7d630cfa185d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:26.473828 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a23aa-ce9d-491c-8407-84c525a9c3fe None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:26.474521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f44b79-6a0d-43d9-af5d-7d630cfa185d None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:26.475077 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f44b79-6a0d-43d9-af5d-7d630cfa185d None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:26.475768 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f44b79-6a0d-43d9-af5d-7d630cfa185d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:26.488082 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f44b79-6a0d-43d9-af5d-7d630cfa185d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:26.489974 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f44b79-6a0d-43d9-af5d-7d630cfa185d None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:26.503454 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-11f44b79-6a0d-43d9-af5d-7d630cfa185d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 14:00:26.503454 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:26.503454 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:26.503454 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:26.503454 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:26.503454 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:26.503454 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:26.503454 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:26.503454 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:26.503454 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:26.503454 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:26.503454 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:26.503454 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:00:26.503454 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:00:26.503454 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:00:26.503454 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:00:26.505338 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:26.505338 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:26.505338 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:00:26.505338 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:26.505338 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:00:26.505338 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:26.505338 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:26.505338 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:26.505338 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:26.505338 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:26.505338 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:26.505338 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:26.505338 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:26.505338 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:26.505338 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:26.505338 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:26.505338 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:26.505338 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:26.506240 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:26.506240 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:26.506240 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:00:26.506240 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:00:26.506240 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:00:26.506240 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:00:26.506240 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:00:26.506240 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:00:26.506240 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 14:00:26.506240 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:26.509346 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 267/558] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Sep 12 14:00:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:26.514770 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f44b79-6a0d-43d9-af5d-7d630cfa185d None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:26.546414 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:26.577122 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737a3b3-0a4f-47f8-8248-9cf9a8a85a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:26.578164 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737a3b3-0a4f-47f8-8248-9cf9a8a85a4a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:26.578661 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737a3b3-0a4f-47f8-8248-9cf9a8a85a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:26.579169 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737a3b3-0a4f-47f8-8248-9cf9a8a85a4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:26.595092 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:26.648066 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f44b79-6a0d-43d9-af5d-7d630cfa185d None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:26.670583 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8737a3b3-0a4f-47f8-8248-9cf9a8a85a4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:26.672073 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8737a3b3-0a4f-47f8-8248-9cf9a8a85a4a None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:26.686336 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f8f2f0-14c9-4698-8647-3660bbddd1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:26.687393 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f8f2f0-14c9-4698-8647-3660bbddd1cd None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:26.687863 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f8f2f0-14c9-4698-8647-3660bbddd1cd None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:26.688370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f8f2f0-14c9-4698-8647-3660bbddd1cd None admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:26.689347 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f8f2f0-14c9-4698-8647-3660bbddd1cd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:26.690005 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f8f2f0-14c9-4698-8647-3660bbddd1cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:26.691796 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f8f2f0-14c9-4698-8647-3660bbddd1cd None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:26.712502 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 268/559] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Sep 12 14:00:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:00:26.713764 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f8f2f0-14c9-4698-8647-3660bbddd1cd None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:26.794779 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 292/560] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:00:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 14:00:26.796709 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737a3b3-0a4f-47f8-8248-9cf9a8a85a4a None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:26.814223 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:26.833189 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f8f2f0-14c9-4698-8647-3660bbddd1cd None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:26.843343 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:26.874111 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985c8a38-4247-44f0-97a7-55281bb58477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:26.875156 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c8a38-4247-44f0-97a7-55281bb58477 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:26.875574 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c8a38-4247-44f0-97a7-55281bb58477 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:26.875992 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c8a38-4247-44f0-97a7-55281bb58477 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:26.887883 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985c8a38-4247-44f0-97a7-55281bb58477 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:26.893150 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-985c8a38-4247-44f0-97a7-55281bb58477 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:00:26.893150 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:26.893150 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:26.893150 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:26.893150 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:26.893150 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:26.893150 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:26.893150 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:26.893150 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:26.893150 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:26.893150 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:26.893150 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:26.893150 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:26.893150 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:26.893150 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:26.893150 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:26.894285 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:26.894285 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:26.894285 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:26.894285 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:26.894285 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:26.894285 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:26.894285 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:26.894285 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:26.894285 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:26.894285 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:26.894285 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:26.894285 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:26.894285 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:26.894285 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:26.894285 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:26.894285 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:26.895352 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:26.895352 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:26.895352 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:26.895352 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:26.895352 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:26.895352 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:26.895352 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:26.895352 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:26.895352 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:26.895352 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:26.895352 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:00:26.895352 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:26.897776 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 269/561] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 12 14:00:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:26.898898 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985c8a38-4247-44f0-97a7-55281bb58477 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:26.908891 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737a3b3-0a4f-47f8-8248-9cf9a8a85a4a None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:26.945582 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb139a-6728-4858-913a-a696e8675c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:26.947093 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb139a-6728-4858-913a-a696e8675c12 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:26.947434 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb139a-6728-4858-913a-a696e8675c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:26.947767 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb139a-6728-4858-913a-a696e8675c12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:26.959449 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb139a-6728-4858-913a-a696e8675c12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:26.961034 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb139a-6728-4858-913a-a696e8675c12 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:26.977593 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-6dfb139a-6728-4858-913a-a696e8675c12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:26.977593 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:26.977593 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:26.977593 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:26.977593 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:26.977593 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:26.977593 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:26.977593 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:26.977593 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:26.977593 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:26.977593 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:26.977593 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:26.977593 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:26.977593 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:26.977593 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:26.977593 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:26.978632 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:26.978632 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:26.978632 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:26.978632 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:26.978632 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:26.978632 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:26.978632 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:26.978632 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:26.978632 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:26.978632 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:26.978632 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:26.978632 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:26.978632 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:26.978632 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:26.978632 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:26.978632 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:26.978632 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:26.979804 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:26.979804 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:26.979804 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:26.979804 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 293/562] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:26.980359 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb139a-6728-4858-913a-a696e8675c12 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:27.026146 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985c8a38-4247-44f0-97a7-55281bb58477 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:27.064291 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e2e3a-99d1-4ec4-bbb7-079c25f7ef69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:27.064925 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e2e3a-99d1-4ec4-bbb7-079c25f7ef69 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:27.065202 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e2e3a-99d1-4ec4-bbb7-079c25f7ef69 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:27.065470 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e2e3a-99d1-4ec4-bbb7-079c25f7ef69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:27.076530 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:27.076780 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562e2e3a-99d1-4ec4-bbb7-079c25f7ef69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:27.078286 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562e2e3a-99d1-4ec4-bbb7-079c25f7ef69 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:27.081205 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:27.088448 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb139a-6728-4858-913a-a696e8675c12 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:27.094756 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-562e2e3a-99d1-4ec4-bbb7-079c25f7ef69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:27.094756 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:27.094756 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:27.094756 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:27.094756 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:27.094756 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:27.094756 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:27.094756 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:27.094756 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:27.094756 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:27.094756 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:27.094756 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:27.094756 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:27.094756 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:27.094756 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:27.094756 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:27.095817 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:27.095817 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:27.095817 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:27.095817 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:27.095817 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:27.095817 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:27.095817 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:27.095817 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:27.095817 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:27.095817 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:27.095817 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:27.095817 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:27.095817 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:27.095817 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:27.095817 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:27.095817 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:27.095817 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:27.096732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:27.096732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:27.096732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:27.096732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 270/563] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:27.096966 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e2e3a-99d1-4ec4-bbb7-079c25f7ef69 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:27.130488 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd143f66-b6cd-43ba-a0f2-698df0d48b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:27.131161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd143f66-b6cd-43ba-a0f2-698df0d48b2e None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:27.131443 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd143f66-b6cd-43ba-a0f2-698df0d48b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:27.131716 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd143f66-b6cd-43ba-a0f2-698df0d48b2e None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:27.132405 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd143f66-b6cd-43ba-a0f2-698df0d48b2e None admin] RBAC: Adding query filter params (name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:27.132708 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd143f66-b6cd-43ba-a0f2-698df0d48b2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:27.134675 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd143f66-b6cd-43ba-a0f2-698df0d48b2e None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:27.144029 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 294/564] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 12 14:00:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:27.145191 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421be636-6d8b-46bf-84ec-23b5315072c4 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:27.145484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421be636-6d8b-46bf-84ec-23b5315072c4 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:27.145760 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421be636-6d8b-46bf-84ec-23b5315072c4 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:27.150915 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 295/565] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:27.150915 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421be636-6d8b-46bf-84ec-23b5315072c4 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:27.218208 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e2e3a-99d1-4ec4-bbb7-079c25f7ef69 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:27.253121 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5929e-ef31-49eb-8a85-9bacfbae9097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:27.253729 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5929e-ef31-49eb-8a85-9bacfbae9097 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:27.254009 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5929e-ef31-49eb-8a85-9bacfbae9097 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:27.254287 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5929e-ef31-49eb-8a85-9bacfbae9097 None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:27.254954 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf5929e-ef31-49eb-8a85-9bacfbae9097 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:27.255256 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf5929e-ef31-49eb-8a85-9bacfbae9097 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:27.256635 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf5929e-ef31-49eb-8a85-9bacfbae9097 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:27.262308 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421be636-6d8b-46bf-84ec-23b5315072c4 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:27.269291 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 271/566] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:27.270077 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5929e-ef31-49eb-8a85-9bacfbae9097 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:27.295147 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315f5e1-27a6-42b5-87ad-dea2e23c4b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:27.295748 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315f5e1-27a6-42b5-87ad-dea2e23c4b10 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:27.296027 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315f5e1-27a6-42b5-87ad-dea2e23c4b10 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:27.296299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315f5e1-27a6-42b5-87ad-dea2e23c4b10 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:27.296944 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c315f5e1-27a6-42b5-87ad-dea2e23c4b10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:27.297242 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c315f5e1-27a6-42b5-87ad-dea2e23c4b10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:27.298660 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c315f5e1-27a6-42b5-87ad-dea2e23c4b10 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:27.310583 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 296/567] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:27.311756 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc7f7c-82c1-4ca9-9515-89cb850e95f0 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:27.312037 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc7f7c-82c1-4ca9-9515-89cb850e95f0 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:27.312307 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc7f7c-82c1-4ca9-9515-89cb850e95f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:27.328528 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:27.330172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-49fc7f7c-82c1-4ca9-9515-89cb850e95f0 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:27.331556 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-49fc7f7c-82c1-4ca9-9515-89cb850e95f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:27.338928 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:27.372283 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5929e-ef31-49eb-8a85-9bacfbae9097 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:27.397099 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 297/568] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:27.398140 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800a499-0bc1-418d-8ea8-81158d7ac28a None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:27.398477 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800a499-0bc1-418d-8ea8-81158d7ac28a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:27.398751 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800a499-0bc1-418d-8ea8-81158d7ac28a None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:27.400053 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 298/569] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:27.400740 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800a499-0bc1-418d-8ea8-81158d7ac28a None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:27.415065 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1582152-4de9-4a12-8363-5fda51a90cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:27.415718 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1582152-4de9-4a12-8363-5fda51a90cbf None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:27.415991 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1582152-4de9-4a12-8363-5fda51a90cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:27.416895 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1582152-4de9-4a12-8363-5fda51a90cbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:27.427393 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1582152-4de9-4a12-8363-5fda51a90cbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:27.428833 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1582152-4de9-4a12-8363-5fda51a90cbf None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:27.447427 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-d1582152-4de9-4a12-8363-5fda51a90cbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:27.447427 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:27.447427 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:27.447427 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:27.447427 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:27.447427 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:27.447427 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:27.447427 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:27.447427 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:27.447427 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:27.447427 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:27.447427 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:27.447427 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:27.447427 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:27.447427 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:27.447427 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:27.448357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:27.448357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:27.448357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:27.448357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:27.448357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:27.448357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:27.448357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:27.448357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:27.448357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:27.448357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:27.448357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:27.448357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:27.448357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:27.448357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:27.448357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:27.448357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:27.448357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:27.449271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:27.449271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:27.449271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:27.449271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 272/570] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:27.449271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1582152-4de9-4a12-8363-5fda51a90cbf None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:27.521029 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800a499-0bc1-418d-8ea8-81158d7ac28a None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:27.557537 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2cc69a-87f9-44f6-b45e-b686b0aa8b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:27.558169 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2cc69a-87f9-44f6-b45e-b686b0aa8b74 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:27.558446 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2cc69a-87f9-44f6-b45e-b686b0aa8b74 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:27.558722 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2cc69a-87f9-44f6-b45e-b686b0aa8b74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:27.565173 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1582152-4de9-4a12-8363-5fda51a90cbf None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:27.565624 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2cc69a-87f9-44f6-b45e-b686b0aa8b74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:27.566693 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2cc69a-87f9-44f6-b45e-b686b0aa8b74 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:27.575419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:27.581042 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-aa2cc69a-87f9-44f6-b45e-b686b0aa8b74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:27.581042 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:27.581042 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:27.581042 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:27.581042 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:27.581042 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:27.581042 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:27.581042 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:27.581042 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:27.581042 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:27.581042 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:27.581042 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:27.581042 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:00:27.581042 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:00:27.581042 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:00:27.581942 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:00:27.581942 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:27.581942 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:27.581942 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:27.581942 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:27.581942 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:27.581942 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:27.581942 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:27.581942 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:27.581942 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:27.581942 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:27.581942 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:27.581942 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:27.581942 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:27.581942 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:27.581942 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:27.581942 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:27.582856 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:00:27.582856 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:00:27.582856 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:00:27.582856 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:00:27.582856 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:00:27.582856 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:00:27.582856 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:27.582856 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:27.587131 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 299/571] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:00:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:27.588346 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151cd48a-be67-43a0-849c-26cc63fb546e None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:27.588636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151cd48a-be67-43a0-849c-26cc63fb546e None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:27.589003 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151cd48a-be67-43a0-849c-26cc63fb546e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:27.596228 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:27.603261 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7122e68d-b4d7-4085-9a1c-8f004eb27044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:27.603886 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7122e68d-b4d7-4085-9a1c-8f004eb27044 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:27.604161 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7122e68d-b4d7-4085-9a1c-8f004eb27044 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:27.604428 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7122e68d-b4d7-4085-9a1c-8f004eb27044 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:27.610680 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-151cd48a-be67-43a0-849c-26cc63fb546e None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:27.611052 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7122e68d-b4d7-4085-9a1c-8f004eb27044 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:27.612102 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-151cd48a-be67-43a0-849c-26cc63fb546e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:27.612469 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7122e68d-b4d7-4085-9a1c-8f004eb27044 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:27.624429 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-7122e68d-b4d7-4085-9a1c-8f004eb27044 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:27.624429 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:27.624429 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:27.624429 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:27.624429 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:27.624429 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:27.624429 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:27.624429 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:27.624429 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:27.624429 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:27.624429 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:27.624429 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:27.624429 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:00:27.624429 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:00:27.624429 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:00:27.625493 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:00:27.625493 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:27.625493 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:27.625493 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:27.625493 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:27.625493 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:27.625493 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:27.625493 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:27.625493 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:27.625493 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:27.625493 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:27.625493 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:27.625493 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:27.625493 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:27.625493 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:27.625493 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:27.625493 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:27.626725 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:00:27.626725 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:00:27.626725 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:00:27.626725 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:00:27.626725 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:00:27.626725 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:00:27.626725 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:27.626725 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:27.626725 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 273/572] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:00:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:27.626725 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7122e68d-b4d7-4085-9a1c-8f004eb27044 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:27.686346 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 300/573] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:27.687212 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151cd48a-be67-43a0-849c-26cc63fb546e None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:27.737315 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7122e68d-b4d7-4085-9a1c-8f004eb27044 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:27.771947 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3140fc8d-31e1-4a43-b684-cd30a3241850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:27.772477 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140fc8d-31e1-4a43-b684-cd30a3241850 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:27.772864 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140fc8d-31e1-4a43-b684-cd30a3241850 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:27.773133 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140fc8d-31e1-4a43-b684-cd30a3241850 None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:27.773874 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3140fc8d-31e1-4a43-b684-cd30a3241850 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:27.774748 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3140fc8d-31e1-4a43-b684-cd30a3241850 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:27.776185 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3140fc8d-31e1-4a43-b684-cd30a3241850 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:27.793218 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 274/574] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:27.793398 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151cd48a-be67-43a0-849c-26cc63fb546e None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:27.793497 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3140fc8d-31e1-4a43-b684-cd30a3241850 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:27.821881 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:27.828760 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:27.838503 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f719ac5-2bba-417b-a821-79d776913007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:27.839219 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f719ac5-2bba-417b-a821-79d776913007 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:27.839563 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f719ac5-2bba-417b-a821-79d776913007 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:27.839896 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f719ac5-2bba-417b-a821-79d776913007 None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:27.840665 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f719ac5-2bba-417b-a821-79d776913007 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:27.841047 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f719ac5-2bba-417b-a821-79d776913007 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:27.842534 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f719ac5-2bba-417b-a821-79d776913007 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:27.859755 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 301/575] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 12 14:00:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:27.860666 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f719ac5-2bba-417b-a821-79d776913007 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:27.900028 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3140fc8d-31e1-4a43-b684-cd30a3241850 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:27.935530 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46208f62-c060-40e1-891b-f5730a7446b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:27.936257 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46208f62-c060-40e1-891b-f5730a7446b0 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:27.936631 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46208f62-c060-40e1-891b-f5730a7446b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:27.936986 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46208f62-c060-40e1-891b-f5730a7446b0 None admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:27.937864 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46208f62-c060-40e1-891b-f5730a7446b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:27.938261 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46208f62-c060-40e1-891b-f5730a7446b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:27.939817 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46208f62-c060-40e1-891b-f5730a7446b0 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:27.956518 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 275/576] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 12 14:00:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:27.956518 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46208f62-c060-40e1-891b-f5730a7446b0 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:27.965259 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f719ac5-2bba-417b-a821-79d776913007 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:27.998053 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfcb5fc-fbb0-443e-9bfc-37e59713a2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:27.998674 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcb5fc-fbb0-443e-9bfc-37e59713a2f6 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:27.998977 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcb5fc-fbb0-443e-9bfc-37e59713a2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:27.999243 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcb5fc-fbb0-443e-9bfc-37e59713a2f6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:28.009000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfcb5fc-fbb0-443e-9bfc-37e59713a2f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:28.014156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-bdfcb5fc-fbb0-443e-9bfc-37e59713a2f6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:00:28.014156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:28.014156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:28.014156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:28.014156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:28.014156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:28.014156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:28.014156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:28.014156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:28.014156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:28.014156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:28.014156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:28.014156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:28.014156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:28.014156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:28.014156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:28.015159 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:28.015159 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:28.015159 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:28.015159 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:28.015159 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:28.015159 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:28.015159 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:28.015159 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:28.015159 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:28.015159 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:28.015159 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:28.015159 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:28.015159 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:28.015159 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:28.015159 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:28.015159 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:28.015992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:28.015992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:28.015992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:28.015992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:28.015992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:28.015992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:28.015992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:28.015992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:28.015992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:28.015992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:28.015992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:00:28.015992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:28.015992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 302/577] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 12 14:00:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:28.020716 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfcb5fc-fbb0-443e-9bfc-37e59713a2f6 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:28.043761 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:28.055911 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46208f62-c060-40e1-891b-f5730a7446b0 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:28.072832 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:28.094593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ba53fd-b176-41ec-9fdd-0d3e5c1876ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:28.095594 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba53fd-b176-41ec-9fdd-0d3e5c1876ab None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:28.095915 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba53fd-b176-41ec-9fdd-0d3e5c1876ab None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:28.096265 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba53fd-b176-41ec-9fdd-0d3e5c1876ab None admin] PATH_INFO: `/v3/users/glance` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:28.104945 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ba53fd-b176-41ec-9fdd-0d3e5c1876ab None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:28.106372 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ba53fd-b176-41ec-9fdd-0d3e5c1876ab None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:28.117371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-c6ba53fd-b176-41ec-9fdd-0d3e5c1876ab None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 12 14:00:28.117371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:28.117371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:28.117371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:28.117371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:28.117371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:28.117371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:28.117371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:28.117371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:28.117371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:28.117371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:28.117371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:28.117371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:00:28.117371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:00:28.117371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:00:28.117371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:00:28.118550 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:28.118550 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:28.118550 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:00:28.118550 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:28.118550 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:00:28.118550 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:28.118550 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:28.118550 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:28.118550 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:28.118550 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:28.118550 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:28.118550 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:28.118550 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:28.118550 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:28.118550 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:28.118550 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:28.118550 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:28.118550 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:28.119729 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:28.119729 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:28.119729 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:00:28.119729 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:00:28.119729 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:00:28.119729 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:00:28.119729 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:00:28.119729 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:00:28.119729 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 12 14:00:28.119729 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:28.119729 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 276/578] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 14:00:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:28.119729 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ba53fd-b176-41ec-9fdd-0d3e5c1876ab None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:28.129507 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfcb5fc-fbb0-443e-9bfc-37e59713a2f6 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:28.163435 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd5e602-e077-4ead-9641-fca22e30d39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:28.164275 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5e602-e077-4ead-9641-fca22e30d39d None admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:28.164575 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5e602-e077-4ead-9641-fca22e30d39d None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:28.165440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5e602-e077-4ead-9641-fca22e30d39d None admin] PATH_INFO: `/v3/projects/f666a617f8d14eb18bbeb076032b82f2/users/854ab2dcec5a48b18f409b33cb61b5da/roles/1c413f4e78ef45b7a821bb884f9cea07` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:28.179237 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd5e602-e077-4ead-9641-fca22e30d39d None admin] RBAC: Authorizing `identity:create_grant(project_id=f666a617f8d14eb18bbeb076032b82f2, user_id=854ab2dcec5a48b18f409b33cb61b5da, role_id=1c413f4e78ef45b7a821bb884f9cea07)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:28.180655 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd5e602-e077-4ead-9641-fca22e30d39d None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:28.192837 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 303/579] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 12 14:00:28 2022] PUT /identity/v3/projects/f666a617f8d14eb18bbeb076032b82f2/users/854ab2dcec5a48b18f409b33cb61b5da/roles/1c413f4e78ef45b7a821bb884f9cea07 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:00:28.193417 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd5e602-e077-4ead-9641-fca22e30d39d None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:28.221390 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ba53fd-b176-41ec-9fdd-0d3e5c1876ab None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:28.257346 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8937451-2e9a-4da5-8466-04b2fe7bec84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:28.257994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8937451-2e9a-4da5-8466-04b2fe7bec84 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:28.258869 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8937451-2e9a-4da5-8466-04b2fe7bec84 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:28.259133 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8937451-2e9a-4da5-8466-04b2fe7bec84 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:28.270671 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8937451-2e9a-4da5-8466-04b2fe7bec84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:28.277001 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-a8937451-2e9a-4da5-8466-04b2fe7bec84 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:00:28.277001 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:28.277001 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:28.277001 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:28.277001 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:28.277001 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:28.277001 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:28.277001 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:28.277001 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:28.277001 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:28.277001 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:28.277001 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:28.277001 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:28.277001 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:28.277001 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:28.277001 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:28.277969 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:28.277969 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:28.277969 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:28.277969 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:28.277969 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:28.277969 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:28.277969 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:28.277969 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:28.277969 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:28.277969 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:28.277969 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:28.277969 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:28.277969 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:28.277969 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:28.277969 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:28.277969 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:28.279773 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:28.279773 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:28.279773 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:28.279773 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:28.279773 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:28.279773 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:28.279773 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:28.279773 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:28.279773 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:28.279773 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:28.279773 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:00:28.279773 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:28.279773 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 277/580] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 12 14:00:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:28.279773 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8937451-2e9a-4da5-8466-04b2fe7bec84 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:28.291852 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:28.311520 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd5e602-e077-4ead-9641-fca22e30d39d None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:28.313422 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:28.356948 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3da5d-363a-4fd4-b3aa-3573ab97ad71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:28.357592 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3da5d-363a-4fd4-b3aa-3573ab97ad71 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:28.358022 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3da5d-363a-4fd4-b3aa-3573ab97ad71 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:28.358419 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3da5d-363a-4fd4-b3aa-3573ab97ad71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:28.359188 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a3da5d-363a-4fd4-b3aa-3573ab97ad71 None admin] RBAC: Adding query filter params (role.id=1c413f4e78ef45b7a821bb884f9cea07, scope.project.id=f666a617f8d14eb18bbeb076032b82f2, user.id=25a87ea9c3c5409b97a35c4857ae3ca6) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:28.359551 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a3da5d-363a-4fd4-b3aa-3573ab97ad71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:28.360972 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a3da5d-363a-4fd4-b3aa-3573ab97ad71 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:28.371768 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 304/581] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 14:00:28 2022] GET /identity/v3/role_assignments?user.id=25a87ea9c3c5409b97a35c4857ae3ca6&scope.project.id=f666a617f8d14eb18bbeb076032b82f2&role.id=1c413f4e78ef45b7a821bb884f9cea07 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:28.371768 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3da5d-363a-4fd4-b3aa-3573ab97ad71 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:28.404186 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8937451-2e9a-4da5-8466-04b2fe7bec84 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:28.451715 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd46d952-5fc2-4891-ac83-7d9899e3604b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:28.452826 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46d952-5fc2-4891-ac83-7d9899e3604b None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:28.453376 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46d952-5fc2-4891-ac83-7d9899e3604b None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:28.453652 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46d952-5fc2-4891-ac83-7d9899e3604b None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:28.454596 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd46d952-5fc2-4891-ac83-7d9899e3604b None admin] RBAC: Adding query filter params (name=service) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:28.454923 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd46d952-5fc2-4891-ac83-7d9899e3604b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:28.457217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd46d952-5fc2-4891-ac83-7d9899e3604b None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:28.469830 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 278/582] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 12 14:00:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:28.470732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd46d952-5fc2-4891-ac83-7d9899e3604b None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:28.510872 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3da5d-363a-4fd4-b3aa-3573ab97ad71 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:28.552956 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec86e6-2fe7-4d58-b4a3-d43c7a325e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:28.554069 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec86e6-2fe7-4d58-b4a3-d43c7a325e5e None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:28.554346 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec86e6-2fe7-4d58-b4a3-d43c7a325e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:28.554613 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec86e6-2fe7-4d58-b4a3-d43c7a325e5e None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:28.556646 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec86e6-2fe7-4d58-b4a3-d43c7a325e5e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:28.556950 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec86e6-2fe7-4d58-b4a3-d43c7a325e5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:28.558089 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:28.558356 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec86e6-2fe7-4d58-b4a3-d43c7a325e5e None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:28.579892 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 305/583] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Sep 12 14:00:28 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:28.581029 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec86e6-2fe7-4d58-b4a3-d43c7a325e5e None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:28.601539 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd46d952-5fc2-4891-ac83-7d9899e3604b None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:28.608956 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:28.648766 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148fd18b-8c03-4496-9bbc-7abda2d61341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:28.650482 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148fd18b-8c03-4496-9bbc-7abda2d61341 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:28.650965 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148fd18b-8c03-4496-9bbc-7abda2d61341 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:28.651405 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148fd18b-8c03-4496-9bbc-7abda2d61341 None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:28.652535 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148fd18b-8c03-4496-9bbc-7abda2d61341 None admin] RBAC: Adding query filter params (name=service) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:28.653007 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148fd18b-8c03-4496-9bbc-7abda2d61341 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:28.655514 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148fd18b-8c03-4496-9bbc-7abda2d61341 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:28.667375 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 279/584] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 12 14:00:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:28.668451 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148fd18b-8c03-4496-9bbc-7abda2d61341 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:28.693754 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec86e6-2fe7-4d58-b4a3-d43c7a325e5e None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:28.735808 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994273e1-115c-4d88-8ff4-1adc14c6e625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:28.736570 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994273e1-115c-4d88-8ff4-1adc14c6e625 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:28.736852 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994273e1-115c-4d88-8ff4-1adc14c6e625 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:28.737119 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994273e1-115c-4d88-8ff4-1adc14c6e625 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:28.748913 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994273e1-115c-4d88-8ff4-1adc14c6e625 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:28.750422 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994273e1-115c-4d88-8ff4-1adc14c6e625 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:28.765117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-994273e1-115c-4d88-8ff4-1adc14c6e625 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:28.765117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:28.765117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:28.765117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:28.765117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:28.765117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:28.765117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:28.765117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:28.765117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:28.765117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:28.765117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:28.765117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:28.765117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:28.765117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:28.765117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:28.765117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:28.766163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:28.766163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:28.766163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:28.766163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:28.766163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:28.766163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:28.766163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:28.766163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:28.766163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:28.766163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:28.766163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:28.766163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:28.766163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:28.766163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:28.766163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:28.766163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:28.766163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:28.767080 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:28.767080 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:28.767080 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:28.770767 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 306/585] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:28.771739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994273e1-115c-4d88-8ff4-1adc14c6e625 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:28.801167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148fd18b-8c03-4496-9bbc-7abda2d61341 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:28.831978 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:28.840804 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4ce95-382f-49f0-a3b3-ef313acee37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:28.841754 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4ce95-382f-49f0-a3b3-ef313acee37f None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:28.842118 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4ce95-382f-49f0-a3b3-ef313acee37f None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:28.842481 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4ce95-382f-49f0-a3b3-ef313acee37f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:28.855119 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d4ce95-382f-49f0-a3b3-ef313acee37f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:28.856664 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d4ce95-382f-49f0-a3b3-ef313acee37f None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:28.871419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-64d4ce95-382f-49f0-a3b3-ef313acee37f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:28.871419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:28.871419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:28.871419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:28.871419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:28.871419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:28.871419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:28.871419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:28.871419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:28.871419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:28.871419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:28.871419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:28.871419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:28.871419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:28.871419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:28.871419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:28.872579 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:28.872579 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:28.872579 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:28.872579 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:28.872579 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:28.872579 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:28.872579 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:28.872579 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:28.872579 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:28.872579 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:28.872579 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:28.872579 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:28.872579 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:28.872579 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:28.872579 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:28.872579 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:28.872579 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:28.873667 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:28.873667 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:28.873667 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:28.875016 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 280/586] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:28.875956 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4ce95-382f-49f0-a3b3-ef313acee37f None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:28.877747 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:28.907773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994273e1-115c-4d88-8ff4-1adc14c6e625 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:28.947708 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38395e1a-b712-4398-9d19-076509d0b3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:28.948350 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38395e1a-b712-4398-9d19-076509d0b3f4 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:28.948627 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38395e1a-b712-4398-9d19-076509d0b3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:28.948999 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38395e1a-b712-4398-9d19-076509d0b3f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:28.962760 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38395e1a-b712-4398-9d19-076509d0b3f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:28.964613 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38395e1a-b712-4398-9d19-076509d0b3f4 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:28.982161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-38395e1a-b712-4398-9d19-076509d0b3f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:28.982161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:28.982161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:28.982161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:28.982161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:28.982161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:28.982161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:28.982161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:28.982161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:28.982161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:28.982161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:28.982161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:28.982161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:28.982161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:28.982161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:28.982161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:28.983168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:28.983168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:28.983168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:28.983168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:28.983168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:28.983168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:28.983168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:28.983168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:28.983168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:28.983168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:28.983168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:28.983168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:28.983168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:28.983168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:28.983168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:28.983168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:28.983168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:28.984052 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:28.984052 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:28.984052 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:28.984052 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 307/587] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:28.984733 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38395e1a-b712-4398-9d19-076509d0b3f4 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:28.984850 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4ce95-382f-49f0-a3b3-ef313acee37f None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:29.020995 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567118ce-bfcb-4515-8710-c606daabf86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:29.021744 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567118ce-bfcb-4515-8710-c606daabf86a None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:29.022408 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567118ce-bfcb-4515-8710-c606daabf86a None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:29.022688 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567118ce-bfcb-4515-8710-c606daabf86a None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:29.023639 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567118ce-bfcb-4515-8710-c606daabf86a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:29.024529 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567118ce-bfcb-4515-8710-c606daabf86a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:29.026496 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567118ce-bfcb-4515-8710-c606daabf86a None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:29.046946 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 281/588] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:29.047942 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567118ce-bfcb-4515-8710-c606daabf86a None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:29.108175 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38395e1a-b712-4398-9d19-076509d0b3f4 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:29.111655 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:29.144734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:29.158260 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4b7475-4f17-4b72-8a20-c42c258602e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:29.159112 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b7475-4f17-4b72-8a20-c42c258602e7 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:29.159394 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b7475-4f17-4b72-8a20-c42c258602e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:29.159662 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b7475-4f17-4b72-8a20-c42c258602e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:29.160823 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4b7475-4f17-4b72-8a20-c42c258602e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:29.161145 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4b7475-4f17-4b72-8a20-c42c258602e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:29.162891 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4b7475-4f17-4b72-8a20-c42c258602e7 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:29.177880 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 308/589] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:29.179210 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4b7475-4f17-4b72-8a20-c42c258602e7 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:29.182594 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567118ce-bfcb-4515-8710-c606daabf86a None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:29.219376 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833deef4-a6c2-4a62-a4cb-9e842dfd79e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:29.220222 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833deef4-a6c2-4a62-a4cb-9e842dfd79e4 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:29.220674 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833deef4-a6c2-4a62-a4cb-9e842dfd79e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:29.221089 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833deef4-a6c2-4a62-a4cb-9e842dfd79e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:29.221900 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833deef4-a6c2-4a62-a4cb-9e842dfd79e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:29.222338 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833deef4-a6c2-4a62-a4cb-9e842dfd79e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:29.224319 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833deef4-a6c2-4a62-a4cb-9e842dfd79e4 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:29.240777 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 282/590] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:29.241778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833deef4-a6c2-4a62-a4cb-9e842dfd79e4 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:29.296462 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4b7475-4f17-4b72-8a20-c42c258602e7 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:29.347610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b769ba2-63d8-40c3-ac62-459b992684b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:29.348485 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b769ba2-63d8-40c3-ac62-459b992684b3 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:29.348764 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b769ba2-63d8-40c3-ac62-459b992684b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:29.349031 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b769ba2-63d8-40c3-ac62-459b992684b3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:29.359879 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b769ba2-63d8-40c3-ac62-459b992684b3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:29.360878 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b769ba2-63d8-40c3-ac62-459b992684b3 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:29.366583 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833deef4-a6c2-4a62-a4cb-9e842dfd79e4 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:29.369937 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:29.376049 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-0b769ba2-63d8-40c3-ac62-459b992684b3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 14:00:29.376049 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:29.376049 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:29.376049 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:29.376049 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:29.376049 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:29.376049 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:29.376049 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:29.376049 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:29.376049 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:29.376049 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:29.376049 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:29.376049 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:00:29.376049 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:00:29.376049 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:00:29.376049 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:00:29.377247 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:29.377247 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:29.377247 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:00:29.377247 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:29.377247 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:00:29.377247 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:29.377247 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:29.377247 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:29.377247 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:29.377247 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:29.377247 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:29.377247 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:29.377247 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:29.377247 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:29.377247 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:29.377247 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:29.377247 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:29.377247 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:29.378168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:29.378168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:29.378168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:00:29.378168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:00:29.378168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:00:29.378168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:00:29.378168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:00:29.378168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:00:29.378168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 14:00:29.378168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:29.382217 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 309/591] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 14:00:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:29.383075 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b769ba2-63d8-40c3-ac62-459b992684b3 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:29.419171 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbde483-9569-4691-8b05-278aa5500687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:29.419807 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbde483-9569-4691-8b05-278aa5500687 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:29.420081 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbde483-9569-4691-8b05-278aa5500687 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:29.420351 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbde483-9569-4691-8b05-278aa5500687 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:29.429967 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbde483-9569-4691-8b05-278aa5500687 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:29.431489 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbde483-9569-4691-8b05-278aa5500687 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:29.443567 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:29.447939 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-ebbde483-9569-4691-8b05-278aa5500687 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:29.447939 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:29.447939 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:29.447939 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:29.447939 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:29.447939 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:29.447939 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:29.447939 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:29.447939 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:29.447939 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:29.447939 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:29.447939 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:29.447939 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:00:29.447939 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:00:29.447939 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:00:29.448832 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:00:29.448832 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:29.448832 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:29.448832 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:29.448832 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:29.448832 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:29.448832 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:29.448832 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:29.448832 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:29.448832 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:29.448832 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:29.448832 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:29.448832 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:29.448832 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:29.448832 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:29.448832 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:29.448832 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:29.449723 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:00:29.449723 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:00:29.449723 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:00:29.449723 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:00:29.449723 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:00:29.449723 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:00:29.449723 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:29.449723 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:29.449723 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 283/592] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:00:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:29.451211 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbde483-9569-4691-8b05-278aa5500687 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:29.512211 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b769ba2-63d8-40c3-ac62-459b992684b3 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:29.561126 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219c16c3-bdff-456e-86b3-61da515bd02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:29.561760 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c16c3-bdff-456e-86b3-61da515bd02f None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:29.562037 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c16c3-bdff-456e-86b3-61da515bd02f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:29.562307 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c16c3-bdff-456e-86b3-61da515bd02f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:29.569265 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbde483-9569-4691-8b05-278aa5500687 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:29.573356 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219c16c3-bdff-456e-86b3-61da515bd02f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:29.574775 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219c16c3-bdff-456e-86b3-61da515bd02f None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:29.590284 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-219c16c3-bdff-456e-86b3-61da515bd02f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 12 14:00:29.590284 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:29.590284 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:29.590284 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:29.590284 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:29.590284 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:29.590284 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:29.590284 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:29.590284 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:29.590284 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:29.590284 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:29.590284 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:29.590284 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:00:29.590284 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:00:29.590284 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:00:29.590284 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:00:29.591290 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:29.591290 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:29.591290 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:00:29.591290 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:29.591290 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:00:29.591290 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:29.591290 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:29.591290 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:29.591290 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:29.591290 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:29.591290 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:29.591290 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:29.591290 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:29.591290 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:29.591290 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:29.591290 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:29.591290 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:29.591290 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:29.592264 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:29.592264 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:29.592264 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:00:29.592264 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:00:29.592264 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:00:29.592264 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:00:29.592264 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:00:29.592264 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:00:29.592264 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 12 14:00:29.592264 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:29.592264 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 310/593] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 12 14:00:29 2022] GET /identity/v3/users/neutron => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:29.595610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219c16c3-bdff-456e-86b3-61da515bd02f None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:29.617102 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb72b2-1b70-480e-a90d-ebb418941014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:29.617874 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb72b2-1b70-480e-a90d-ebb418941014 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:29.618005 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb72b2-1b70-480e-a90d-ebb418941014 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:29.618327 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb72b2-1b70-480e-a90d-ebb418941014 None admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:29.619527 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bb72b2-1b70-480e-a90d-ebb418941014 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:29.619888 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bb72b2-1b70-480e-a90d-ebb418941014 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:29.621655 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bb72b2-1b70-480e-a90d-ebb418941014 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:29.645992 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 284/594] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Sep 12 14:00:29 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:29.651648 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb72b2-1b70-480e-a90d-ebb418941014 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:29.686113 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:29.723356 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219c16c3-bdff-456e-86b3-61da515bd02f None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:29.736986 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:29.772459 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a1780d-d1c5-4343-b278-db160fae05f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:29.773065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1780d-d1c5-4343-b278-db160fae05f5 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:29.773342 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1780d-d1c5-4343-b278-db160fae05f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:29.773623 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1780d-d1c5-4343-b278-db160fae05f5 None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:29.775390 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a1780d-d1c5-4343-b278-db160fae05f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:29.775702 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a1780d-d1c5-4343-b278-db160fae05f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:29.777115 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a1780d-d1c5-4343-b278-db160fae05f5 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:29.793363 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb72b2-1b70-480e-a90d-ebb418941014 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:29.794255 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 311/595] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 12 14:00:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:29.796433 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a1780d-d1c5-4343-b278-db160fae05f5 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:29.853625 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7149cefa-fe5f-49d6-961b-0198ba554b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:29.854245 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149cefa-fe5f-49d6-961b-0198ba554b6c None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:29.854525 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149cefa-fe5f-49d6-961b-0198ba554b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:29.854799 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149cefa-fe5f-49d6-961b-0198ba554b6c None admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:29.855554 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7149cefa-fe5f-49d6-961b-0198ba554b6c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:29.855856 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7149cefa-fe5f-49d6-961b-0198ba554b6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:29.857794 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7149cefa-fe5f-49d6-961b-0198ba554b6c None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:29.881851 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 285/596] 10.222.0.42 () {56 vars in 1139 bytes} [Mon Sep 12 14:00:29 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:29.882376 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7149cefa-fe5f-49d6-961b-0198ba554b6c None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:29.922655 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a1780d-d1c5-4343-b278-db160fae05f5 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:29.964198 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938ede7-57fd-469d-8feb-a3e3f7680bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:29.964837 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938ede7-57fd-469d-8feb-a3e3f7680bf2 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:29.965117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938ede7-57fd-469d-8feb-a3e3f7680bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:29.965387 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938ede7-57fd-469d-8feb-a3e3f7680bf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:29.979109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1938ede7-57fd-469d-8feb-a3e3f7680bf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:29.981330 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1938ede7-57fd-469d-8feb-a3e3f7680bf2 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:29.982360 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:30.003377 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7149cefa-fe5f-49d6-961b-0198ba554b6c None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:30.004099 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-1938ede7-57fd-469d-8feb-a3e3f7680bf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:30.004099 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:30.004099 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:30.004099 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:30.004099 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:30.004099 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:30.004099 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:30.004099 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:30.004099 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:30.004099 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:30.004099 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:30.004099 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:30.004099 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:30.004099 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:30.004099 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:30.004099 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:30.005066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:30.005066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:30.005066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:30.005066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:30.005066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:30.005066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:30.005066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:30.005066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:30.005066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:30.005066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:30.005066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:30.005066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:30.005066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:30.005066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:30.005066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:30.005066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:30.005066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:30.006151 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:30.006151 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:30.006151 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:30.007031 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 312/597] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:30.010872 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938ede7-57fd-469d-8feb-a3e3f7680bf2 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:30.037518 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:30.041610 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd9921-491b-4463-9d39-3447912f3c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:30.042386 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd9921-491b-4463-9d39-3447912f3c1a None admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:30.042876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd9921-491b-4463-9d39-3447912f3c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:30.043252 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd9921-491b-4463-9d39-3447912f3c1a None admin] PATH_INFO: `/v3/projects/f666a617f8d14eb18bbeb076032b82f2/users/2092a9639067461c9a837a891f56889b/roles/1c413f4e78ef45b7a821bb884f9cea07` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:30.059995 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bd9921-491b-4463-9d39-3447912f3c1a None admin] RBAC: Authorizing `identity:create_grant(project_id=f666a617f8d14eb18bbeb076032b82f2, user_id=2092a9639067461c9a837a891f56889b, role_id=1c413f4e78ef45b7a821bb884f9cea07)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:30.061690 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bd9921-491b-4463-9d39-3447912f3c1a None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:30.083068 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 286/598] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 12 14:00:29 2022] PUT /identity/v3/projects/f666a617f8d14eb18bbeb076032b82f2/users/2092a9639067461c9a837a891f56889b/roles/1c413f4e78ef45b7a821bb884f9cea07 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:00:30.085994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd9921-491b-4463-9d39-3447912f3c1a None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:30.170934 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938ede7-57fd-469d-8feb-a3e3f7680bf2 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:30.230576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3140bdf-5aaf-41e6-bd1e-021d2e36b017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:30.231654 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3140bdf-5aaf-41e6-bd1e-021d2e36b017 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:30.232317 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3140bdf-5aaf-41e6-bd1e-021d2e36b017 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:30.232745 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3140bdf-5aaf-41e6-bd1e-021d2e36b017 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:30.260663 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3140bdf-5aaf-41e6-bd1e-021d2e36b017 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:30.262557 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3140bdf-5aaf-41e6-bd1e-021d2e36b017 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:30.293519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-e3140bdf-5aaf-41e6-bd1e-021d2e36b017 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:30.293519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:30.293519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:30.293519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:30.293519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:30.293519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:30.293519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:30.293519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:30.293519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:30.293519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:30.293519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:30.293519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:30.293519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:30.293519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:30.293519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:30.293519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:30.294527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:30.294527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:30.294527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:30.294527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:30.294527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:30.294527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:30.294527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:30.294527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:30.294527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:30.294527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:30.294527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:30.294527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:30.294527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:30.294527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:30.294527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:30.294527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:30.294527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:30.295741 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:30.295741 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:30.295741 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:30.295741 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 313/599] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:30.300721 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3140bdf-5aaf-41e6-bd1e-021d2e36b017 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:30.315088 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd9921-491b-4463-9d39-3447912f3c1a None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:30.350560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:30.358001 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300675bf-bfc8-405f-b8d1-fb3a83a030fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:30.359348 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300675bf-bfc8-405f-b8d1-fb3a83a030fb None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:30.359730 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300675bf-bfc8-405f-b8d1-fb3a83a030fb None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:30.360281 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300675bf-bfc8-405f-b8d1-fb3a83a030fb None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:30.361132 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300675bf-bfc8-405f-b8d1-fb3a83a030fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:30.361632 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300675bf-bfc8-405f-b8d1-fb3a83a030fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:30.363240 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300675bf-bfc8-405f-b8d1-fb3a83a030fb None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:30.377118 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 287/600] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:30.382640 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300675bf-bfc8-405f-b8d1-fb3a83a030fb None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:30.393310 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:30.432198 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3140bdf-5aaf-41e6-bd1e-021d2e36b017 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:30.498045 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee846a-98cf-45b5-b624-71e07f52f169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:30.499799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee846a-98cf-45b5-b624-71e07f52f169 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:30.501803 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee846a-98cf-45b5-b624-71e07f52f169 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:30.502338 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee846a-98cf-45b5-b624-71e07f52f169 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:30.503482 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee846a-98cf-45b5-b624-71e07f52f169 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:30.506102 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee846a-98cf-45b5-b624-71e07f52f169 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:30.509785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee846a-98cf-45b5-b624-71e07f52f169 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:30.534872 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 314/601] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:30.541099 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee846a-98cf-45b5-b624-71e07f52f169 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:30.570299 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300675bf-bfc8-405f-b8d1-fb3a83a030fb None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:30.606148 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dd71bc-6bdc-4bdb-834a-2690adebe163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:30.606785 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd71bc-6bdc-4bdb-834a-2690adebe163 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:30.607112 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd71bc-6bdc-4bdb-834a-2690adebe163 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:30.607454 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd71bc-6bdc-4bdb-834a-2690adebe163 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:30.614505 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dd71bc-6bdc-4bdb-834a-2690adebe163 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:30.616258 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dd71bc-6bdc-4bdb-834a-2690adebe163 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:30.628143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-a5dd71bc-6bdc-4bdb-834a-2690adebe163 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:30.628143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:30.628143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:30.628143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:30.628143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:30.628143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:30.628143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:30.628143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:30.628143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:30.628143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:30.628143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:30.628143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:30.628143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:00:30.628143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:00:30.628143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:00:30.629521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:00:30.629521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:30.629521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:30.629521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:30.629521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:30.629521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:30.629521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:30.629521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:30.629521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:30.629521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:30.629521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:30.629521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:30.629521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:30.629521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:30.629521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:30.629521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:30.629521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:30.630444 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:00:30.630444 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:00:30.630444 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:00:30.630444 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:00:30.630444 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:00:30.630444 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:00:30.630444 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:30.630444 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:30.630444 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 288/602] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:00:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:30.632094 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588780ad-4547-48e6-9f45-7b7a86d66abf None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:30.632375 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588780ad-4547-48e6-9f45-7b7a86d66abf None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:30.632645 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588780ad-4547-48e6-9f45-7b7a86d66abf None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:30.633927 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 289/603] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:30.634981 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b7a78-ad5f-4549-b9f8-6ab0eba798de None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:30.635341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b7a78-ad5f-4549-b9f8-6ab0eba798de None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:30.635515 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b7a78-ad5f-4549-b9f8-6ab0eba798de None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:30.640574 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 290/604] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:30.640574 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b7a78-ad5f-4549-b9f8-6ab0eba798de None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:30.673270 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee846a-98cf-45b5-b624-71e07f52f169 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:30.691257 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:30.706614 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:30.712576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7091006f-7177-4df0-9d8d-fe9303f9cadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:30.713991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7091006f-7177-4df0-9d8d-fe9303f9cadc None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:30.714275 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7091006f-7177-4df0-9d8d-fe9303f9cadc None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:30.714547 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7091006f-7177-4df0-9d8d-fe9303f9cadc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:30.722487 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7091006f-7177-4df0-9d8d-fe9303f9cadc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:30.723957 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7091006f-7177-4df0-9d8d-fe9303f9cadc None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:30.736185 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-7091006f-7177-4df0-9d8d-fe9303f9cadc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:30.736185 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:30.736185 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:30.736185 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:30.736185 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:30.736185 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:30.736185 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:30.736185 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:30.736185 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:30.736185 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:30.736185 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:30.736185 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:30.736185 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:00:30.736185 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:00:30.736185 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:00:30.737087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:00:30.737087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:30.737087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:30.737087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:30.737087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:30.737087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:30.737087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:30.737087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:30.737087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:30.737087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:30.737087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:30.737087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:30.737087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:30.737087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:30.737087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:30.737087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:30.737087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:30.743833 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:00:30.743833 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:00:30.743833 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:00:30.743833 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:00:30.743833 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:00:30.743833 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:00:30.743833 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:30.743833 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:30.743833 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 315/605] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:00:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:30.743833 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30845e-922a-4e84-96ee-2ccaee1bcf7b None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:30.743833 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30845e-922a-4e84-96ee-2ccaee1bcf7b None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:30.744434 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30845e-922a-4e84-96ee-2ccaee1bcf7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:30.753590 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b7a78-ad5f-4549-b9f8-6ab0eba798de None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:30.760465 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-ff30845e-922a-4e84-96ee-2ccaee1bcf7b None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:30.762044 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ff30845e-922a-4e84-96ee-2ccaee1bcf7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:30.787513 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c858b179-b98e-43e6-a3f1-0ff4c9beaca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:30.788115 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858b179-b98e-43e6-a3f1-0ff4c9beaca0 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:30.788387 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858b179-b98e-43e6-a3f1-0ff4c9beaca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:30.788654 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858b179-b98e-43e6-a3f1-0ff4c9beaca0 None admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:30.789332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c858b179-b98e-43e6-a3f1-0ff4c9beaca0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:30.790235 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c858b179-b98e-43e6-a3f1-0ff4c9beaca0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:30.791681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c858b179-b98e-43e6-a3f1-0ff4c9beaca0 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:30.808908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 291/606] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 12 14:00:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:30.808908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b5db6-27ba-45e8-8f6d-bddbd9c248ca None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:30.808908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b5db6-27ba-45e8-8f6d-bddbd9c248ca None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:30.808908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b5db6-27ba-45e8-8f6d-bddbd9c248ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:30.827553 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-806b5db6-27ba-45e8-8f6d-bddbd9c248ca None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:30.828811 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-806b5db6-27ba-45e8-8f6d-bddbd9c248ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:30.846795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 316/607] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:30.854503 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff30845e-922a-4e84-96ee-2ccaee1bcf7b None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:30.904293 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 292/608] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:30.911787 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806b5db6-27ba-45e8-8f6d-bddbd9c248ca None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:30.971903 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff30845e-922a-4e84-96ee-2ccaee1bcf7b None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:30.999610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:31.010467 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:31.019219 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e97f83-1137-463b-b070-9449162e3663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:31.020074 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e97f83-1137-463b-b070-9449162e3663 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:31.021862 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e97f83-1137-463b-b070-9449162e3663 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:31.022277 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e97f83-1137-463b-b070-9449162e3663 None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:31.023221 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e97f83-1137-463b-b070-9449162e3663 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:31.024987 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e97f83-1137-463b-b070-9449162e3663 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:31.026911 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e97f83-1137-463b-b070-9449162e3663 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:31.045127 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 317/609] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 12 14:00:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:31.046558 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e97f83-1137-463b-b070-9449162e3663 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:31.138810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806b5db6-27ba-45e8-8f6d-bddbd9c248ca None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:31.152370 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e97f83-1137-463b-b070-9449162e3663 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:31.189549 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ee26c-7c35-4888-b40d-fd755e7a699a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:31.190632 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ee26c-7c35-4888-b40d-fd755e7a699a None admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:31.191210 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ee26c-7c35-4888-b40d-fd755e7a699a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:31.191686 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ee26c-7c35-4888-b40d-fd755e7a699a None admin] PATH_INFO: `/v3/projects/f666a617f8d14eb18bbeb076032b82f2/users/865b1647468c4efe8efe4febe2203a58/roles/1c413f4e78ef45b7a821bb884f9cea07` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:31.208777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536ee26c-7c35-4888-b40d-fd755e7a699a None admin] RBAC: Authorizing `identity:create_grant(project_id=f666a617f8d14eb18bbeb076032b82f2, user_id=865b1647468c4efe8efe4febe2203a58, role_id=1c413f4e78ef45b7a821bb884f9cea07)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:31.210288 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536ee26c-7c35-4888-b40d-fd755e7a699a None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:31.212891 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93567aa8-dfc9-4531-87b5-765647967296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:31.213634 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93567aa8-dfc9-4531-87b5-765647967296 None admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:31.213923 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93567aa8-dfc9-4531-87b5-765647967296 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:31.214202 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93567aa8-dfc9-4531-87b5-765647967296 None admin] PATH_INFO: `/v3/projects/f666a617f8d14eb18bbeb076032b82f2/users/8da38bc8dd2f4cf798649ce96097a4b8/roles/1c413f4e78ef45b7a821bb884f9cea07` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:31.232627 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93567aa8-dfc9-4531-87b5-765647967296 None admin] RBAC: Authorizing `identity:create_grant(project_id=f666a617f8d14eb18bbeb076032b82f2, user_id=8da38bc8dd2f4cf798649ce96097a4b8, role_id=1c413f4e78ef45b7a821bb884f9cea07)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:31.234022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93567aa8-dfc9-4531-87b5-765647967296 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:31.235018 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 318/610] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 12 14:00:31 2022] PUT /identity/v3/projects/f666a617f8d14eb18bbeb076032b82f2/users/865b1647468c4efe8efe4febe2203a58/roles/1c413f4e78ef45b7a821bb884f9cea07 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:00:31.236055 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ee26c-7c35-4888-b40d-fd755e7a699a None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:31.252547 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 293/611] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 12 14:00:30 2022] PUT /identity/v3/projects/f666a617f8d14eb18bbeb076032b82f2/users/8da38bc8dd2f4cf798649ce96097a4b8/roles/1c413f4e78ef45b7a821bb884f9cea07 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:00:31.274900 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93567aa8-dfc9-4531-87b5-765647967296 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:31.297933 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:31.407460 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ee26c-7c35-4888-b40d-fd755e7a699a None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:31.453904 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47180131-8aea-4c1c-a3fc-6987b6765e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:31.454801 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:31.456569 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47180131-8aea-4c1c-a3fc-6987b6765e92 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:31.457139 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47180131-8aea-4c1c-a3fc-6987b6765e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:31.457613 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47180131-8aea-4c1c-a3fc-6987b6765e92 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:31.474111 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47180131-8aea-4c1c-a3fc-6987b6765e92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:31.480848 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-47180131-8aea-4c1c-a3fc-6987b6765e92 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:00:31.480848 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:31.480848 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:31.480848 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:31.480848 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:31.480848 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:31.480848 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:31.480848 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:31.480848 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:31.480848 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:31.480848 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:31.480848 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:31.480848 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:31.480848 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:31.480848 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:31.480848 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:31.481879 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:31.481879 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:31.481879 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:31.481879 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:31.481879 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:31.481879 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:31.481879 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:31.481879 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:31.481879 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:31.481879 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:31.481879 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:31.481879 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:31.481879 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:31.481879 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:31.481879 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:31.481879 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:31.483294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:31.483294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:31.483294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:31.483294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:31.483294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:31.483294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:31.483294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:31.483294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:31.483294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:31.483294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:31.483294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:00:31.483294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:31.483294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 319/612] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 12 14:00:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:31.485977 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93567aa8-dfc9-4531-87b5-765647967296 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:31.487862 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47180131-8aea-4c1c-a3fc-6987b6765e92 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:31.538792 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c80e145-8367-4abf-b00e-a27240e868dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:31.539872 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80e145-8367-4abf-b00e-a27240e868dc None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:31.540354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80e145-8367-4abf-b00e-a27240e868dc None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:31.541255 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80e145-8367-4abf-b00e-a27240e868dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:31.553951 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c80e145-8367-4abf-b00e-a27240e868dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:31.561212 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-8c80e145-8367-4abf-b00e-a27240e868dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:00:31.561212 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:31.561212 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:31.561212 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:31.561212 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:31.561212 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:31.561212 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:31.561212 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:31.561212 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:31.561212 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:31.561212 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:31.561212 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:31.561212 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:31.561212 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:31.561212 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:31.561212 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:31.562136 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:31.562136 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:31.562136 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:31.562136 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:31.562136 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:31.562136 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:31.562136 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:31.562136 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:31.562136 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:31.562136 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:31.562136 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:31.562136 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:31.562136 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:31.562136 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:31.562136 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:31.562136 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:31.562995 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:31.562995 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:31.562995 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:31.562995 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:31.562995 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:31.562995 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:31.562995 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:31.562995 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:31.562995 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:31.562995 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:31.562995 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:00:31.562995 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:31.565207 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 294/613] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 12 14:00:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:00:31.571420 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c80e145-8367-4abf-b00e-a27240e868dc None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:31.603369 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:31.611236 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47180131-8aea-4c1c-a3fc-6987b6765e92 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:31.651437 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c325971-e354-4cae-9e7b-19c8a09e149c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:31.652440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c325971-e354-4cae-9e7b-19c8a09e149c None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:31.652921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c325971-e354-4cae-9e7b-19c8a09e149c None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:31.653394 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c325971-e354-4cae-9e7b-19c8a09e149c None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:31.654367 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c325971-e354-4cae-9e7b-19c8a09e149c None admin] RBAC: Adding query filter params (name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:31.654892 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c325971-e354-4cae-9e7b-19c8a09e149c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:31.656477 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c325971-e354-4cae-9e7b-19c8a09e149c None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:31.673238 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 320/614] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 12 14:00:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:00:31.678426 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c325971-e354-4cae-9e7b-19c8a09e149c None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:31.688153 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c80e145-8367-4abf-b00e-a27240e868dc None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:31.726806 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e58cb-0f58-45b6-9d2b-9df2e19ece79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:31.727490 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e58cb-0f58-45b6-9d2b-9df2e19ece79 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:31.727768 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e58cb-0f58-45b6-9d2b-9df2e19ece79 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:31.728039 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e58cb-0f58-45b6-9d2b-9df2e19ece79 None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:31.728705 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e58cb-0f58-45b6-9d2b-9df2e19ece79 None admin] RBAC: Adding query filter params (name=service) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:31.729002 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e58cb-0f58-45b6-9d2b-9df2e19ece79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:31.733837 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e58cb-0f58-45b6-9d2b-9df2e19ece79 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:31.740396 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 295/615] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 12 14:00:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:31.746482 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e58cb-0f58-45b6-9d2b-9df2e19ece79 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:31.767209 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:31.860670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e58cb-0f58-45b6-9d2b-9df2e19ece79 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:31.880518 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c325971-e354-4cae-9e7b-19c8a09e149c None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:31.902872 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d861f-f7f0-4a4c-a78c-440520e57f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:31.902872 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d861f-f7f0-4a4c-a78c-440520e57f55 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:31.902872 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d861f-f7f0-4a4c-a78c-440520e57f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:31.902872 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d861f-f7f0-4a4c-a78c-440520e57f55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:31.914095 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44d861f-f7f0-4a4c-a78c-440520e57f55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:31.916384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44d861f-f7f0-4a4c-a78c-440520e57f55 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:31.932602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-b44d861f-f7f0-4a4c-a78c-440520e57f55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:31.932602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:31.932602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:31.932602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:31.932602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:31.932602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:31.932602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:31.932602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:31.932602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:31.932602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:31.932602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:31.932602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:31.932602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:31.932602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:31.932602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:31.932602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:31.933553 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:31.933553 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:31.933553 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:31.933553 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:31.933553 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:31.933553 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:31.933553 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:31.933553 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:31.933553 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:31.933553 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:31.933553 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:31.933553 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:31.933553 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:31.933553 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:31.933553 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:31.933553 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:31.933553 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:31.934440 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:31.934440 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:31.934440 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:31.934440 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 296/616] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:31.939940 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d861f-f7f0-4a4c-a78c-440520e57f55 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:31.978695 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112881e9-4b71-4025-af1a-60d7ba382004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:31.979747 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112881e9-4b71-4025-af1a-60d7ba382004 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:31.980236 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112881e9-4b71-4025-af1a-60d7ba382004 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:31.980700 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112881e9-4b71-4025-af1a-60d7ba382004 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:32.009564 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112881e9-4b71-4025-af1a-60d7ba382004 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:32.011255 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:32.011562 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112881e9-4b71-4025-af1a-60d7ba382004 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:32.047481 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d861f-f7f0-4a4c-a78c-440520e57f55 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:32.052499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-112881e9-4b71-4025-af1a-60d7ba382004 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:32.052499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:32.052499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:32.052499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:32.052499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:32.052499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:32.052499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:32.052499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:32.052499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:32.052499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:32.052499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:32.052499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:32.052499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:32.052499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:32.052499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:32.052499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:32.053395 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:32.053395 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:32.053395 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:32.053395 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:32.053395 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:32.053395 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:32.053395 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:32.053395 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:32.053395 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:32.053395 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:32.053395 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:32.053395 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:32.053395 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:32.053395 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:32.053395 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:32.053395 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:32.053395 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:32.054265 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:32.054265 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:32.054265 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:32.056614 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 321/617] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:00:32.067816 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112881e9-4b71-4025-af1a-60d7ba382004 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:32.088025 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:32.093129 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df877033-a67a-48e2-b0c7-045fb3b8d306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:32.093129 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df877033-a67a-48e2-b0c7-045fb3b8d306 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:32.093129 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df877033-a67a-48e2-b0c7-045fb3b8d306 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:32.093129 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df877033-a67a-48e2-b0c7-045fb3b8d306 None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:32.093618 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df877033-a67a-48e2-b0c7-045fb3b8d306 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:32.093618 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df877033-a67a-48e2-b0c7-045fb3b8d306 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:32.093749 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df877033-a67a-48e2-b0c7-045fb3b8d306 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:32.108059 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 297/618] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:32.113770 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df877033-a67a-48e2-b0c7-045fb3b8d306 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:32.237436 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df877033-a67a-48e2-b0c7-045fb3b8d306 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:32.295274 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:32.315582 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b4bd2-b2d9-4189-94d3-d5db5ae36cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:32.316234 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b4bd2-b2d9-4189-94d3-d5db5ae36cea None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:32.316509 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b4bd2-b2d9-4189-94d3-d5db5ae36cea None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:32.316776 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b4bd2-b2d9-4189-94d3-d5db5ae36cea None admin] PATH_INFO: `/v3/users/swift` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:32.335870 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1b4bd2-b2d9-4189-94d3-d5db5ae36cea None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:32.337389 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1b4bd2-b2d9-4189-94d3-d5db5ae36cea None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:32.340821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112881e9-4b71-4025-af1a-60d7ba382004 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:32.360810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-1f1b4bd2-b2d9-4189-94d3-d5db5ae36cea None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 12 14:00:32.360810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:32.360810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:32.360810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:32.360810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:32.360810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:32.360810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:32.360810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:32.360810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:32.360810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:32.360810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:32.360810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:32.360810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:00:32.360810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:00:32.360810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:00:32.360810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:00:32.361950 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:32.361950 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:32.361950 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:00:32.361950 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:32.361950 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:00:32.361950 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:32.361950 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:32.361950 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:32.361950 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:32.361950 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:32.361950 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:32.361950 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:32.361950 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:32.361950 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:32.361950 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:32.361950 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:32.361950 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:32.361950 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:32.362934 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:32.362934 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:32.362934 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:00:32.362934 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:00:32.362934 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:00:32.362934 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:00:32.362934 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:00:32.362934 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:00:32.362934 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 12 14:00:32.362934 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:32.368929 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 298/619] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 12 14:00:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:32.369768 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b4bd2-b2d9-4189-94d3-d5db5ae36cea None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:32.396642 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1618e7bb-1d37-491e-b4c7-09fce23945c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:32.397256 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618e7bb-1d37-491e-b4c7-09fce23945c2 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:32.397535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618e7bb-1d37-491e-b4c7-09fce23945c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:32.397959 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618e7bb-1d37-491e-b4c7-09fce23945c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:32.398607 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1618e7bb-1d37-491e-b4c7-09fce23945c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:32.399124 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1618e7bb-1d37-491e-b4c7-09fce23945c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:32.400740 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1618e7bb-1d37-491e-b4c7-09fce23945c2 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:32.414387 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 322/620] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:32.420610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1618e7bb-1d37-491e-b4c7-09fce23945c2 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:32.497169 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:32.537030 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1618e7bb-1d37-491e-b4c7-09fce23945c2 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:32.573720 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a68cf95-143d-4e8f-a451-951457a30658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:32.574275 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a68cf95-143d-4e8f-a451-951457a30658 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:32.574553 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a68cf95-143d-4e8f-a451-951457a30658 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:32.574824 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a68cf95-143d-4e8f-a451-951457a30658 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:32.582630 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a68cf95-143d-4e8f-a451-951457a30658 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:32.584099 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a68cf95-143d-4e8f-a451-951457a30658 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:32.608469 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-8a68cf95-143d-4e8f-a451-951457a30658 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 14:00:32.608469 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:32.608469 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:32.608469 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:32.608469 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:32.608469 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:32.608469 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:32.608469 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:32.608469 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:32.608469 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:32.608469 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:32.608469 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:32.608469 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:00:32.608469 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:00:32.608469 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:00:32.608469 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:00:32.609475 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:32.609475 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:32.609475 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:00:32.609475 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:32.609475 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:00:32.609475 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:32.609475 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:32.609475 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:32.609475 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:32.609475 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:32.609475 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:32.609475 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:32.609475 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:32.609475 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:32.609475 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:32.609475 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:32.609475 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:32.609475 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:32.610390 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:32.610390 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:32.610390 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:00:32.610390 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:00:32.610390 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:00:32.610390 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:00:32.610390 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:00:32.610390 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:00:32.610390 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 14:00:32.610390 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:32.616141 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b4bd2-b2d9-4189-94d3-d5db5ae36cea None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:32.618086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 323/621] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Sep 12 14:00:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:32.632048 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a68cf95-143d-4e8f-a451-951457a30658 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:32.723552 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb6a8cf-7607-46d0-803c-283e9d4e509b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:32.724879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6a8cf-7607-46d0-803c-283e9d4e509b None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:32.725356 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6a8cf-7607-46d0-803c-283e9d4e509b None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:32.725824 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6a8cf-7607-46d0-803c-283e9d4e509b None admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:32.726749 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb6a8cf-7607-46d0-803c-283e9d4e509b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:32.727342 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb6a8cf-7607-46d0-803c-283e9d4e509b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:32.728994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb6a8cf-7607-46d0-803c-283e9d4e509b None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:32.740165 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a68cf95-143d-4e8f-a451-951457a30658 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:32.770997 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 299/622] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Sep 12 14:00:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:32.772269 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac6c47-21cc-4378-bb03-0cd6aa2ea7e2 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:32.772554 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac6c47-21cc-4378-bb03-0cd6aa2ea7e2 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:32.772827 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac6c47-21cc-4378-bb03-0cd6aa2ea7e2 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:32.774865 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 300/623] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:32.777412 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbdf5bc-1b7c-4091-a0fd-735344b7102b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:32.778023 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbdf5bc-1b7c-4091-a0fd-735344b7102b None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:32.778301 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbdf5bc-1b7c-4091-a0fd-735344b7102b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:32.778571 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbdf5bc-1b7c-4091-a0fd-735344b7102b None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:32.779442 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbdf5bc-1b7c-4091-a0fd-735344b7102b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:32.779720 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbdf5bc-1b7c-4091-a0fd-735344b7102b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:32.780465 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ac6c47-21cc-4378-bb03-0cd6aa2ea7e2 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:32.781239 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbdf5bc-1b7c-4091-a0fd-735344b7102b None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:32.792431 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:32.805299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 324/624] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Sep 12 14:00:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:32.808187 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0c9cf-2bd6-4baa-8098-9c6c7e2cf61b None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:32.808472 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0c9cf-2bd6-4baa-8098-9c6c7e2cf61b None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:32.808749 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0c9cf-2bd6-4baa-8098-9c6c7e2cf61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:32.832189 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-b5a0c9cf-2bd6-4baa-8098-9c6c7e2cf61b None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:32.833320 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b5a0c9cf-2bd6-4baa-8098-9c6c7e2cf61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:32.835259 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:32.895524 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ac6c47-21cc-4378-bb03-0cd6aa2ea7e2 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:32.918209 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 325/625] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:32.919840 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a0c9cf-2bd6-4baa-8098-9c6c7e2cf61b None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:32.931891 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d44b006-4e25-4697-b18d-bb91ec12aef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:32.932651 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44b006-4e25-4697-b18d-bb91ec12aef4 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:32.933026 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44b006-4e25-4697-b18d-bb91ec12aef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:32.933395 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44b006-4e25-4697-b18d-bb91ec12aef4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:32.944264 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d44b006-4e25-4697-b18d-bb91ec12aef4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:32.946400 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d44b006-4e25-4697-b18d-bb91ec12aef4 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:32.962336 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-4d44b006-4e25-4697-b18d-bb91ec12aef4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:32.962336 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:32.962336 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:32.962336 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:32.962336 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:32.962336 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:32.962336 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:32.962336 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:32.962336 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:32.962336 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:32.962336 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:32.962336 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:32.962336 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:32.962336 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:32.962336 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:32.962336 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:32.963372 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:32.963372 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:32.963372 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:32.963372 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:32.963372 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:32.963372 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:32.963372 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:32.963372 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:32.963372 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:32.963372 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:32.963372 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:32.963372 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:32.963372 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:32.963372 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:32.963372 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:32.963372 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:32.963372 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:32.964268 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:32.964268 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:32.964268 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:32.964268 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 301/626] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:32.972040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d44b006-4e25-4697-b18d-bb91ec12aef4 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:33.082689 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a0c9cf-2bd6-4baa-8098-9c6c7e2cf61b None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:33.107378 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:33.129019 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:33.139779 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d44b006-4e25-4697-b18d-bb91ec12aef4 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:33.144065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392cafef-c15a-43b5-989b-bb2fa8568c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:33.144675 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392cafef-c15a-43b5-989b-bb2fa8568c9a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:33.144952 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392cafef-c15a-43b5-989b-bb2fa8568c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:33.145222 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392cafef-c15a-43b5-989b-bb2fa8568c9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:33.213812 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4e550-937c-4ed0-b0eb-c0bc7571ee82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:33.214415 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4e550-937c-4ed0-b0eb-c0bc7571ee82 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:33.214689 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4e550-937c-4ed0-b0eb-c0bc7571ee82 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:33.214982 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4e550-937c-4ed0-b0eb-c0bc7571ee82 None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:33.215682 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d4e550-937c-4ed0-b0eb-c0bc7571ee82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:33.215985 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d4e550-937c-4ed0-b0eb-c0bc7571ee82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:33.218172 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d4e550-937c-4ed0-b0eb-c0bc7571ee82 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:33.242889 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392cafef-c15a-43b5-989b-bb2fa8568c9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:33.242889 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392cafef-c15a-43b5-989b-bb2fa8568c9a None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:33.251236 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 302/627] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:00:33.252353 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4e550-937c-4ed0-b0eb-c0bc7571ee82 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:33.378618 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 326/628] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:00:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 14:00:33.378618 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392cafef-c15a-43b5-989b-bb2fa8568c9a None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:33.398045 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:33.411055 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4e550-937c-4ed0-b0eb-c0bc7571ee82 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:33.454027 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b6c0b-1535-41d6-acb6-6e0d01b1e98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:33.454449 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b6c0b-1535-41d6-acb6-6e0d01b1e98a None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:33.454776 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b6c0b-1535-41d6-acb6-6e0d01b1e98a None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:33.455157 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b6c0b-1535-41d6-acb6-6e0d01b1e98a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:33.464466 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923b6c0b-1535-41d6-acb6-6e0d01b1e98a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:33.466062 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923b6c0b-1535-41d6-acb6-6e0d01b1e98a None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:33.480409 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-923b6c0b-1535-41d6-acb6-6e0d01b1e98a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:33.480409 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:33.480409 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:33.480409 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:33.480409 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:33.480409 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:33.480409 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:33.480409 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:33.480409 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:33.480409 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:33.480409 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:33.480409 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:33.480409 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:00:33.480409 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:00:33.480409 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:00:33.481350 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:00:33.481350 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:33.481350 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:33.481350 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:33.481350 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:33.481350 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:33.481350 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:33.481350 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:33.481350 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:33.481350 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:33.481350 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:33.481350 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:33.481350 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:33.481350 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:33.481350 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:33.481350 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:33.481350 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:33.482247 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:00:33.482247 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:00:33.482247 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:00:33.482247 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:00:33.482247 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:00:33.482247 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:00:33.482247 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:33.482247 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:33.486714 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 303/629] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:00:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:33.487943 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b6c0b-1535-41d6-acb6-6e0d01b1e98a None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:33.509009 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:33.515752 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392cafef-c15a-43b5-989b-bb2fa8568c9a None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:33.562106 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56387f6-4f82-4d6c-b819-50b2d8986c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:33.564016 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56387f6-4f82-4d6c-b819-50b2d8986c68 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:33.564552 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56387f6-4f82-4d6c-b819-50b2d8986c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:33.565232 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56387f6-4f82-4d6c-b819-50b2d8986c68 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:33.581566 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56387f6-4f82-4d6c-b819-50b2d8986c68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:33.589086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-f56387f6-4f82-4d6c-b819-50b2d8986c68 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:00:33.589086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:33.589086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:33.589086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:33.589086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:33.589086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:33.589086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:33.589086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:33.589086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:33.589086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:33.589086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:33.589086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:33.589086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:33.589086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:33.589086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:33.589086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:33.590266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:33.590266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:33.590266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:33.590266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:33.590266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:33.590266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:33.590266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:33.590266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:33.590266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:33.590266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:33.590266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:33.590266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:33.590266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:33.590266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:33.590266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:33.590266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:33.591320 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:33.591320 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:33.591320 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:33.591320 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:33.591320 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:33.591320 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:33.591320 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:33.591320 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:33.591320 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:33.591320 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:33.591320 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:00:33.591320 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:33.593687 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 327/630] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 12 14:00:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:33.594674 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56387f6-4f82-4d6c-b819-50b2d8986c68 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:33.629720 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b6c0b-1535-41d6-acb6-6e0d01b1e98a None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:33.674439 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2efd8c-2b8f-41f1-abe2-6b1e7041552a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:33.675859 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2efd8c-2b8f-41f1-abe2-6b1e7041552a None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:33.676347 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2efd8c-2b8f-41f1-abe2-6b1e7041552a None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:33.676886 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2efd8c-2b8f-41f1-abe2-6b1e7041552a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:33.689309 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2efd8c-2b8f-41f1-abe2-6b1e7041552a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:33.690777 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2efd8c-2b8f-41f1-abe2-6b1e7041552a None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:33.695472 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:33.706435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-dc2efd8c-2b8f-41f1-abe2-6b1e7041552a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:33.706435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:33.706435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:33.706435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:33.706435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:33.706435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:33.706435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:33.706435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:33.706435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:33.706435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:33.706435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:33.706435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:33.706435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:33.706435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:33.706435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:33.706435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:33.707588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:33.707588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:33.707588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:33.707588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:33.707588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:33.707588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:33.707588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:33.707588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:33.707588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:33.707588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:33.707588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:33.707588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:33.707588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:33.707588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:33.707588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:33.707588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:33.707588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:33.708516 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:33.708516 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:33.708516 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:33.708516 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 304/631] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:33.708516 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2efd8c-2b8f-41f1-abe2-6b1e7041552a None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:33.729789 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56387f6-4f82-4d6c-b819-50b2d8986c68 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:33.766165 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea7af1-3dc4-46b7-9e98-1f2f8e155256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:33.766792 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea7af1-3dc4-46b7-9e98-1f2f8e155256 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:33.767100 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea7af1-3dc4-46b7-9e98-1f2f8e155256 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:33.767374 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea7af1-3dc4-46b7-9e98-1f2f8e155256 None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:33.768080 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea7af1-3dc4-46b7-9e98-1f2f8e155256 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:33.768382 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea7af1-3dc4-46b7-9e98-1f2f8e155256 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:33.770058 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea7af1-3dc4-46b7-9e98-1f2f8e155256 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:33.780559 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:33.787513 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 328/632] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 12 14:00:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:33.789195 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e184d-cd2b-45cc-ba48-4f2cc8acda6a None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:33.790341 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e184d-cd2b-45cc-ba48-4f2cc8acda6a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:33.790808 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e184d-cd2b-45cc-ba48-4f2cc8acda6a None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:33.793341 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 329/633] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:33.795776 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579e184d-cd2b-45cc-ba48-4f2cc8acda6a None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:33.818718 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2efd8c-2b8f-41f1-abe2-6b1e7041552a None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:33.858761 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71cd26c-2deb-45d1-b23a-8c32f3d79838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:33.859830 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cd26c-2deb-45d1-b23a-8c32f3d79838 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:33.860324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cd26c-2deb-45d1-b23a-8c32f3d79838 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:33.860794 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cd26c-2deb-45d1-b23a-8c32f3d79838 None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:33.861793 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71cd26c-2deb-45d1-b23a-8c32f3d79838 None admin] RBAC: Adding query filter params (name=service) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:33.862381 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71cd26c-2deb-45d1-b23a-8c32f3d79838 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:33.864029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71cd26c-2deb-45d1-b23a-8c32f3d79838 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:33.873996 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 305/634] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 12 14:00:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:33.874796 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71cd26c-2deb-45d1-b23a-8c32f3d79838 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:33.900995 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579e184d-cd2b-45cc-ba48-4f2cc8acda6a None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:33.934174 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a92fdb9-d7a2-4006-9c40-5a26a4313b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:33.934777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a92fdb9-d7a2-4006-9c40-5a26a4313b0f None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:33.935091 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a92fdb9-d7a2-4006-9c40-5a26a4313b0f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:33.935357 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a92fdb9-d7a2-4006-9c40-5a26a4313b0f None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:33.935991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a92fdb9-d7a2-4006-9c40-5a26a4313b0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:33.936289 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a92fdb9-d7a2-4006-9c40-5a26a4313b0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:33.937686 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a92fdb9-d7a2-4006-9c40-5a26a4313b0f None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:33.939210 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:33.950930 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 330/635] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:33.952167 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664f399-9dbd-4b6e-b01d-e080e02b3aec None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:33.952447 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664f399-9dbd-4b6e-b01d-e080e02b3aec None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:33.952720 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664f399-9dbd-4b6e-b01d-e080e02b3aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:33.972776 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-d664f399-9dbd-4b6e-b01d-e080e02b3aec None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:33.973972 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d664f399-9dbd-4b6e-b01d-e080e02b3aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:33.974117 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71cd26c-2deb-45d1-b23a-8c32f3d79838 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:34.003715 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:34.012883 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10050fdd-3805-4879-b8d5-5e2628775b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:34.013529 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10050fdd-3805-4879-b8d5-5e2628775b65 None admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:34.013803 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10050fdd-3805-4879-b8d5-5e2628775b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:34.014071 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10050fdd-3805-4879-b8d5-5e2628775b65 None admin] PATH_INFO: `/v3/projects/f666a617f8d14eb18bbeb076032b82f2/users/25a87ea9c3c5409b97a35c4857ae3ca6/roles/1c413f4e78ef45b7a821bb884f9cea07` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:34.028758 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10050fdd-3805-4879-b8d5-5e2628775b65 None admin] RBAC: Authorizing `identity:create_grant(project_id=f666a617f8d14eb18bbeb076032b82f2, user_id=25a87ea9c3c5409b97a35c4857ae3ca6, role_id=1c413f4e78ef45b7a821bb884f9cea07)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:34.030176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10050fdd-3805-4879-b8d5-5e2628775b65 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:34.043647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 306/636] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 12 14:00:33 2022] PUT /identity/v3/projects/f666a617f8d14eb18bbeb076032b82f2/users/25a87ea9c3c5409b97a35c4857ae3ca6/roles/1c413f4e78ef45b7a821bb884f9cea07 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:00:34.044210 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 331/637] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:34.044427 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10050fdd-3805-4879-b8d5-5e2628775b65 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:34.045904 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ae878-912b-477e-bcc9-41156a3487ee None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:34.046193 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ae878-912b-477e-bcc9-41156a3487ee None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:34.046469 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ae878-912b-477e-bcc9-41156a3487ee None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:34.047967 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 332/638] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:34.052943 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ae878-912b-477e-bcc9-41156a3487ee None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:34.170025 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10050fdd-3805-4879-b8d5-5e2628775b65 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:34.184046 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ae878-912b-477e-bcc9-41156a3487ee None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:34.210384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba471e18-da68-4196-a235-4c6b25d0d041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:34.211880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba471e18-da68-4196-a235-4c6b25d0d041 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:34.212353 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba471e18-da68-4196-a235-4c6b25d0d041 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:34.212782 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba471e18-da68-4196-a235-4c6b25d0d041 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:34.214409 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:34.225576 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba471e18-da68-4196-a235-4c6b25d0d041 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:34.226499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a93a7c-b33d-43fb-98fc-20621fd483fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:34.227186 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a93a7c-b33d-43fb-98fc-20621fd483fd None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:34.227466 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a93a7c-b33d-43fb-98fc-20621fd483fd None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:34.227751 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a93a7c-b33d-43fb-98fc-20621fd483fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:34.228042 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba471e18-da68-4196-a235-4c6b25d0d041 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:34.235838 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a93a7c-b33d-43fb-98fc-20621fd483fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:34.237758 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a93a7c-b33d-43fb-98fc-20621fd483fd None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:34.244577 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-ba471e18-da68-4196-a235-4c6b25d0d041 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:34.244577 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:34.244577 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:34.244577 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:34.244577 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:34.244577 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:34.244577 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:34.244577 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:34.244577 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:34.244577 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:34.244577 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:34.244577 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:34.244577 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:34.244577 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:34.244577 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:34.244577 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:34.245833 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:34.245833 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:34.245833 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:34.245833 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:34.245833 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:34.245833 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:34.245833 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:34.245833 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:34.245833 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:34.245833 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:34.245833 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:34.245833 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:34.245833 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:34.245833 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:34.245833 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:34.245833 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:34.245833 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:34.246943 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:34.246943 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:34.246943 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:34.248683 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 307/639] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:34.249982 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e83ece-dccb-428f-942f-449269260eee None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:34.250151 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-f5a93a7c-b33d-43fb-98fc-20621fd483fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:34.250151 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:34.250151 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:34.250151 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:34.250151 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:34.250151 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:34.250151 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:34.250151 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:34.250151 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:34.250151 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:34.250151 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:34.250151 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:34.250151 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:00:34.250151 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:00:34.250151 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:00:34.251192 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:00:34.251192 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:34.251192 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:34.251192 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:34.251192 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:34.251192 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:34.251192 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:34.251192 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:34.251192 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:34.251192 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:34.251192 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:34.251192 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:34.251192 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:34.251192 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:34.251192 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:34.251192 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:34.251192 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:34.252370 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:00:34.252370 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:00:34.252370 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:00:34.252370 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:00:34.252370 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:00:34.252370 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:00:34.252370 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:34.252370 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:34.253216 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e83ece-dccb-428f-942f-449269260eee None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:34.253636 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e83ece-dccb-428f-942f-449269260eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:34.255476 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 333/640] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:00:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:34.256366 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a93a7c-b33d-43fb-98fc-20621fd483fd None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:34.273264 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-a2e83ece-dccb-428f-942f-449269260eee None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:34.275322 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-a2e83ece-dccb-428f-942f-449269260eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:34.310004 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:34.375375 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 308/641] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:34.384332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e83ece-dccb-428f-942f-449269260eee None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:34.388402 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a93a7c-b33d-43fb-98fc-20621fd483fd None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:34.435608 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376dd4c-2087-477b-a332-1335839c94a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:34.436138 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376dd4c-2087-477b-a332-1335839c94a3 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:34.436416 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376dd4c-2087-477b-a332-1335839c94a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:34.436689 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376dd4c-2087-477b-a332-1335839c94a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:34.437444 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4376dd4c-2087-477b-a332-1335839c94a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:34.437983 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4376dd4c-2087-477b-a332-1335839c94a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:34.440305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4376dd4c-2087-477b-a332-1335839c94a3 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:34.460785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 334/642] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:34.460785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376dd4c-2087-477b-a332-1335839c94a3 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:34.509183 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:34.510069 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e83ece-dccb-428f-942f-449269260eee None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:34.548143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4418ced-cde7-4e9f-ae12-d44595dd6662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:34.548815 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4418ced-cde7-4e9f-ae12-d44595dd6662 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:34.549097 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4418ced-cde7-4e9f-ae12-d44595dd6662 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:34.549427 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4418ced-cde7-4e9f-ae12-d44595dd6662 None admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:34.550132 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4418ced-cde7-4e9f-ae12-d44595dd6662 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:34.550418 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4418ced-cde7-4e9f-ae12-d44595dd6662 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:34.551942 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4418ced-cde7-4e9f-ae12-d44595dd6662 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:34.568873 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 309/643] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 12 14:00:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:34.568873 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4418ced-cde7-4e9f-ae12-d44595dd6662 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:34.576272 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376dd4c-2087-477b-a332-1335839c94a3 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:34.592752 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:34.619615 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c5957-86fb-4ec8-af14-ac7507772ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:34.620255 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c5957-86fb-4ec8-af14-ac7507772ef8 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:34.620535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c5957-86fb-4ec8-af14-ac7507772ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:34.620808 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c5957-86fb-4ec8-af14-ac7507772ef8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:34.634733 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c5957-86fb-4ec8-af14-ac7507772ef8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:34.641259 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-409c5957-86fb-4ec8-af14-ac7507772ef8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:00:34.641259 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:34.641259 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:34.641259 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:34.641259 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:34.641259 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:34.641259 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:34.641259 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:34.641259 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:34.641259 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:34.641259 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:34.641259 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:34.641259 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:34.641259 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:34.641259 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:34.641259 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:34.642269 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:34.642269 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:34.642269 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:34.642269 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:34.642269 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:34.642269 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:34.642269 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:34.642269 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:34.642269 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:34.642269 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:34.642269 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:34.642269 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:34.642269 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:34.642269 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:34.642269 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:34.642269 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:34.643167 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:34.643167 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:34.643167 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:34.643167 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:34.643167 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:34.643167 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:34.643167 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:34.643167 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:34.643167 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:34.643167 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:34.643167 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:00:34.643167 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:34.643167 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 335/644] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 12 14:00:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:34.643858 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c5957-86fb-4ec8-af14-ac7507772ef8 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:34.687011 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4418ced-cde7-4e9f-ae12-d44595dd6662 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:34.724485 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc4004-f9f7-4656-9507-472f5292546f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:34.725121 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc4004-f9f7-4656-9507-472f5292546f None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:34.725393 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc4004-f9f7-4656-9507-472f5292546f None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:34.725663 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc4004-f9f7-4656-9507-472f5292546f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:34.734656 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc4004-f9f7-4656-9507-472f5292546f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:34.736125 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc4004-f9f7-4656-9507-472f5292546f None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:34.747610 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-51fc4004-f9f7-4656-9507-472f5292546f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 12 14:00:34.747610 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:34.747610 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:34.747610 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:34.747610 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:34.747610 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:34.747610 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:34.747610 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:34.747610 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:34.747610 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:34.747610 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:34.747610 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:34.747610 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:00:34.747610 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:00:34.747610 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:00:34.747610 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:00:34.748560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:34.748560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:34.748560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:00:34.748560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:34.748560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:00:34.748560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:34.748560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:34.748560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:34.748560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:34.748560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:34.748560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:34.748560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:34.748560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:34.748560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:34.748560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:34.748560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:34.748560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:34.748560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:34.750047 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:34.750047 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:34.750047 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:00:34.750047 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:00:34.750047 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:00:34.750047 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:00:34.750047 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:00:34.750047 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:00:34.750047 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 12 14:00:34.750047 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:34.750047 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 310/645] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 14:00:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:34.750047 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc4004-f9f7-4656-9507-472f5292546f None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:34.758654 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c5957-86fb-4ec8-af14-ac7507772ef8 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:34.762019 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:34.798888 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e48d5-560c-4ca8-a2ca-11b7cd66155f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:34.799500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e48d5-560c-4ca8-a2ca-11b7cd66155f None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:34.799779 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e48d5-560c-4ca8-a2ca-11b7cd66155f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:34.800046 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e48d5-560c-4ca8-a2ca-11b7cd66155f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:34.800776 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e48d5-560c-4ca8-a2ca-11b7cd66155f None admin] RBAC: Adding query filter params (role.id=1c413f4e78ef45b7a821bb884f9cea07, scope.project.id=f666a617f8d14eb18bbeb076032b82f2, user.id=854ab2dcec5a48b18f409b33cb61b5da) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:34.801080 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e48d5-560c-4ca8-a2ca-11b7cd66155f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:34.802871 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e48d5-560c-4ca8-a2ca-11b7cd66155f None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:34.815366 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 336/646] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 14:00:34 2022] GET /identity/v3/role_assignments?user.id=854ab2dcec5a48b18f409b33cb61b5da&scope.project.id=f666a617f8d14eb18bbeb076032b82f2&role.id=1c413f4e78ef45b7a821bb884f9cea07 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:34.816745 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e48d5-560c-4ca8-a2ca-11b7cd66155f None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:34.845444 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:34.871013 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc4004-f9f7-4656-9507-472f5292546f None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:34.909661 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f561d96-960c-4d7a-a0b7-812056da8ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:34.910310 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f561d96-960c-4d7a-a0b7-812056da8ffa None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:34.910581 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f561d96-960c-4d7a-a0b7-812056da8ffa None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:34.910941 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f561d96-960c-4d7a-a0b7-812056da8ffa None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:34.911559 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f561d96-960c-4d7a-a0b7-812056da8ffa None admin] RBAC: Adding query filter params (name=service) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:34.911860 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f561d96-960c-4d7a-a0b7-812056da8ffa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:34.913305 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f561d96-960c-4d7a-a0b7-812056da8ffa None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:34.923318 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 311/647] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 12 14:00:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:34.924290 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f561d96-960c-4d7a-a0b7-812056da8ffa None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:34.978649 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e48d5-560c-4ca8-a2ca-11b7cd66155f None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:35.022087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd3050-da16-4f44-afe4-542d7e7e6b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:35.023272 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd3050-da16-4f44-afe4-542d7e7e6b64 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:35.024568 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd3050-da16-4f44-afe4-542d7e7e6b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:35.025355 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd3050-da16-4f44-afe4-542d7e7e6b64 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:35.039971 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafd3050-da16-4f44-afe4-542d7e7e6b64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:35.044705 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f561d96-960c-4d7a-a0b7-812056da8ffa None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:35.045396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-dafd3050-da16-4f44-afe4-542d7e7e6b64 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:00:35.045396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:35.045396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:35.045396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:35.045396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:35.045396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:35.045396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:35.045396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:35.045396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:35.045396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:35.045396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:35.045396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:35.045396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:35.045396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:35.045396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:35.045396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:35.046319 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:35.046319 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:35.046319 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:35.046319 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:35.046319 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:35.046319 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:35.046319 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:35.046319 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:35.046319 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:35.046319 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:35.046319 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:35.046319 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:35.046319 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:35.046319 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:35.046319 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:35.046319 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:35.048719 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:35.048719 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:35.048719 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:35.048719 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:35.048719 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:35.048719 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:35.048719 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:35.048719 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:35.048719 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:35.048719 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:35.048719 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:00:35.048719 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:35.048719 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 337/648] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 12 14:00:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:35.048719 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd3050-da16-4f44-afe4-542d7e7e6b64 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:35.084212 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:35.093542 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f352aa-95c1-4b9b-b85a-b839c4e49b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:35.094503 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f352aa-95c1-4b9b-b85a-b839c4e49b69 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:35.095926 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f352aa-95c1-4b9b-b85a-b839c4e49b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:35.096413 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f352aa-95c1-4b9b-b85a-b839c4e49b69 None admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:35.097365 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f352aa-95c1-4b9b-b85a-b839c4e49b69 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:35.097928 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f352aa-95c1-4b9b-b85a-b839c4e49b69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:35.099659 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f352aa-95c1-4b9b-b85a-b839c4e49b69 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:35.119621 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:35.124189 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 312/649] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Sep 12 14:00:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:35.128949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f352aa-95c1-4b9b-b85a-b839c4e49b69 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:35.172130 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd3050-da16-4f44-afe4-542d7e7e6b64 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:35.208665 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05eb4f-e27f-4fcb-a5f0-0814708a75f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:35.209396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05eb4f-e27f-4fcb-a5f0-0814708a75f2 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:35.209656 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05eb4f-e27f-4fcb-a5f0-0814708a75f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:35.209932 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05eb4f-e27f-4fcb-a5f0-0814708a75f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:35.221296 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a05eb4f-e27f-4fcb-a5f0-0814708a75f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:35.222772 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a05eb4f-e27f-4fcb-a5f0-0814708a75f2 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:35.238779 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-2a05eb4f-e27f-4fcb-a5f0-0814708a75f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:35.238779 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:35.238779 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:35.238779 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:35.238779 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:35.238779 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:35.238779 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:35.238779 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:35.238779 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:35.238779 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:35.238779 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:35.238779 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:35.238779 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:35.238779 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:35.238779 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:35.238779 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:35.239784 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:35.239784 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:35.239784 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:35.239784 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:35.239784 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:35.239784 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:35.239784 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:35.239784 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:35.239784 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:35.239784 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:35.239784 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:35.239784 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:35.239784 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:35.239784 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:35.239784 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:35.239784 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:35.239784 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:35.240657 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:35.240657 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:35.240657 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:35.240657 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 338/650] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:35.240883 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05eb4f-e27f-4fcb-a5f0-0814708a75f2 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:35.257127 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f352aa-95c1-4b9b-b85a-b839c4e49b69 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:35.304346 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5a1194-0a1d-4942-9366-9a83919ab140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:35.305110 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a1194-0a1d-4942-9366-9a83919ab140 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:35.305389 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a1194-0a1d-4942-9366-9a83919ab140 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:35.305667 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a1194-0a1d-4942-9366-9a83919ab140 None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:35.307007 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5a1194-0a1d-4942-9366-9a83919ab140 None admin] RBAC: Adding query filter params (name=service) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:35.307318 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5a1194-0a1d-4942-9366-9a83919ab140 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:35.309215 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5a1194-0a1d-4942-9366-9a83919ab140 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:35.319500 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 313/651] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 12 14:00:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:35.320409 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5a1194-0a1d-4942-9366-9a83919ab140 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:35.331715 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:35.351684 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05eb4f-e27f-4fcb-a5f0-0814708a75f2 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:35.388748 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65453d82-51c7-4dce-876c-a85c04b793d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:35.389357 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65453d82-51c7-4dce-876c-a85c04b793d2 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:35.389633 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65453d82-51c7-4dce-876c-a85c04b793d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:35.389901 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65453d82-51c7-4dce-876c-a85c04b793d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:35.392919 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:35.438735 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5a1194-0a1d-4942-9366-9a83919ab140 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:35.458981 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65453d82-51c7-4dce-876c-a85c04b793d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:35.460527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65453d82-51c7-4dce-876c-a85c04b793d2 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:35.476616 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4048769-627e-4126-b8f0-dc22536dbd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:35.478111 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4048769-627e-4126-b8f0-dc22536dbd00 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:35.478558 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4048769-627e-4126-b8f0-dc22536dbd00 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:35.479068 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4048769-627e-4126-b8f0-dc22536dbd00 None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:35.479858 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4048769-627e-4126-b8f0-dc22536dbd00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:35.480465 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4048769-627e-4126-b8f0-dc22536dbd00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:35.482086 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4048769-627e-4126-b8f0-dc22536dbd00 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:35.500743 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 314/652] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:35.501937 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4048769-627e-4126-b8f0-dc22536dbd00 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:35.543744 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:35.563397 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 339/653] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:00:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 14:00:35.563993 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65453d82-51c7-4dce-876c-a85c04b793d2 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:35.611142 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4048769-627e-4126-b8f0-dc22536dbd00 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:35.635978 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:35.650993 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb4592-54d6-4df6-8c97-df96aa816689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:35.651835 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb4592-54d6-4df6-8c97-df96aa816689 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:35.652284 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb4592-54d6-4df6-8c97-df96aa816689 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:35.652762 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb4592-54d6-4df6-8c97-df96aa816689 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:35.661728 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65453d82-51c7-4dce-876c-a85c04b793d2 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:35.664318 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb4592-54d6-4df6-8c97-df96aa816689 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:35.666445 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb4592-54d6-4df6-8c97-df96aa816689 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:35.682708 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-a3fb4592-54d6-4df6-8c97-df96aa816689 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:35.682708 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:35.682708 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:35.682708 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:35.682708 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:35.682708 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:35.682708 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:35.682708 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:35.682708 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:35.682708 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:35.682708 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:35.682708 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:35.682708 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:35.682708 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:35.682708 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:35.682708 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:35.683986 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:35.683986 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:35.683986 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:35.683986 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:35.683986 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:35.683986 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:35.683986 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:35.683986 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:35.683986 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:35.683986 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:35.683986 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:35.683986 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:35.683986 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:35.683986 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:35.683986 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:35.683986 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:35.683986 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:35.685021 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:35.685021 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:35.685021 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:35.686757 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 315/654] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:35.687933 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb4592-54d6-4df6-8c97-df96aa816689 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:35.697100 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e6eca-d42c-4655-9a4f-6b48a8b41e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:35.697732 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e6eca-d42c-4655-9a4f-6b48a8b41e75 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:35.698019 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e6eca-d42c-4655-9a4f-6b48a8b41e75 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:35.698267 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e6eca-d42c-4655-9a4f-6b48a8b41e75 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:35.705700 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00e6eca-d42c-4655-9a4f-6b48a8b41e75 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:35.707160 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00e6eca-d42c-4655-9a4f-6b48a8b41e75 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:35.719460 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-c00e6eca-d42c-4655-9a4f-6b48a8b41e75 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 14:00:35.719460 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:35.719460 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:35.719460 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:35.719460 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:35.719460 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:35.719460 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:35.719460 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:35.719460 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:35.719460 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:35.719460 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:35.719460 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:35.719460 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:00:35.719460 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:00:35.719460 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:00:35.719460 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:00:35.720420 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:35.720420 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:35.720420 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:00:35.720420 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:35.720420 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:00:35.720420 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:35.720420 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:35.720420 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:35.720420 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:35.720420 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:35.720420 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:35.720420 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:35.720420 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:35.720420 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:35.720420 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:35.720420 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:35.720420 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:35.720420 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:35.721361 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:35.721361 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:35.721361 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:00:35.721361 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:00:35.721361 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:00:35.721361 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:00:35.721361 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:00:35.721361 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:00:35.721361 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 14:00:35.721361 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:35.725234 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 340/655] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 14:00:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:35.726103 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e6eca-d42c-4655-9a4f-6b48a8b41e75 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:35.768681 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:35.807097 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb4592-54d6-4df6-8c97-df96aa816689 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:35.827228 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e6eca-d42c-4655-9a4f-6b48a8b41e75 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:35.852275 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa7e47b-b570-44d2-a8c0-d83d0ddba3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:35.853785 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7e47b-b570-44d2-a8c0-d83d0ddba3ac None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:35.854263 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7e47b-b570-44d2-a8c0-d83d0ddba3ac None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:35.854730 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7e47b-b570-44d2-a8c0-d83d0ddba3ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:35.861370 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78aed1-8e1f-4ea0-9e1f-b5d18845ac56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:35.861973 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78aed1-8e1f-4ea0-9e1f-b5d18845ac56 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:35.862252 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78aed1-8e1f-4ea0-9e1f-b5d18845ac56 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:35.862519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78aed1-8e1f-4ea0-9e1f-b5d18845ac56 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:35.863200 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd78aed1-8e1f-4ea0-9e1f-b5d18845ac56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:35.863496 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd78aed1-8e1f-4ea0-9e1f-b5d18845ac56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:35.864832 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd78aed1-8e1f-4ea0-9e1f-b5d18845ac56 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:35.868385 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa7e47b-b570-44d2-a8c0-d83d0ddba3ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:35.870068 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa7e47b-b570-44d2-a8c0-d83d0ddba3ac None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:35.878216 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 341/656] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:35.878810 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78aed1-8e1f-4ea0-9e1f-b5d18845ac56 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:35.888152 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-daa7e47b-b570-44d2-a8c0-d83d0ddba3ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:35.888152 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:35.888152 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:35.888152 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:35.888152 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:35.888152 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:35.888152 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:35.888152 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:35.888152 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:35.888152 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:35.888152 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:35.888152 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:35.888152 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:35.888152 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:35.888152 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:35.888152 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:35.889167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:35.889167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:35.889167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:35.889167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:35.889167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:35.889167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:35.889167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:35.889167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:35.889167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:35.889167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:35.889167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:35.889167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:35.889167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:35.889167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:35.889167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:35.889167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:35.889167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:35.890063 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:35.890063 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:35.890063 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:35.892369 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 316/657] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:35.893499 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa7e47b-b570-44d2-a8c0-d83d0ddba3ac None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:35.924194 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:35.977700 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:35.980519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78aed1-8e1f-4ea0-9e1f-b5d18845ac56 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:36.008999 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa7e47b-b570-44d2-a8c0-d83d0ddba3ac None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:36.016476 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e158c8-02f9-4bbd-9746-5fe93611a969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:36.017417 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e158c8-02f9-4bbd-9746-5fe93611a969 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:36.017417 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e158c8-02f9-4bbd-9746-5fe93611a969 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:36.017613 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e158c8-02f9-4bbd-9746-5fe93611a969 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:36.018303 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e158c8-02f9-4bbd-9746-5fe93611a969 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:36.018600 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e158c8-02f9-4bbd-9746-5fe93611a969 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:36.020272 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e158c8-02f9-4bbd-9746-5fe93611a969 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:36.037392 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 342/658] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Sep 12 14:00:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:36.038206 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e158c8-02f9-4bbd-9746-5fe93611a969 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:36.048443 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17285353-9f7e-4969-8a7d-8de8b7629737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:36.049386 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17285353-9f7e-4969-8a7d-8de8b7629737 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:36.049864 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17285353-9f7e-4969-8a7d-8de8b7629737 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:36.050332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17285353-9f7e-4969-8a7d-8de8b7629737 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:36.059504 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17285353-9f7e-4969-8a7d-8de8b7629737 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:36.061298 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17285353-9f7e-4969-8a7d-8de8b7629737 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:36.074476 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-17285353-9f7e-4969-8a7d-8de8b7629737 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 12 14:00:36.074476 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:36.074476 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:36.074476 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:36.074476 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:36.074476 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:36.074476 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:36.074476 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:36.074476 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:36.074476 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:36.074476 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:36.074476 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:36.074476 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:00:36.074476 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:00:36.074476 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:00:36.074476 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:00:36.075436 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:36.075436 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:36.075436 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:00:36.075436 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:36.075436 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:00:36.075436 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:36.075436 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:36.075436 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:36.075436 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:36.075436 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:36.075436 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:36.075436 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:36.075436 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:36.075436 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:36.075436 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:36.075436 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:36.075436 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:36.075436 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:36.076361 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:36.076361 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:36.076361 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:00:36.076361 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:00:36.076361 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:00:36.076361 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:00:36.076361 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:00:36.076361 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:00:36.076361 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 12 14:00:36.076361 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:36.078918 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 317/659] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 12 14:00:35 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:36.079893 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17285353-9f7e-4969-8a7d-8de8b7629737 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:36.142257 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e158c8-02f9-4bbd-9746-5fe93611a969 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:36.172274 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:36.179535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c639c9-80cd-477d-97e7-36341be73a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:36.180800 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c639c9-80cd-477d-97e7-36341be73a0f None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:36.181079 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c639c9-80cd-477d-97e7-36341be73a0f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:36.181345 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c639c9-80cd-477d-97e7-36341be73a0f None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:36.182265 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c639c9-80cd-477d-97e7-36341be73a0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:36.182372 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c639c9-80cd-477d-97e7-36341be73a0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:36.183739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c639c9-80cd-477d-97e7-36341be73a0f None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:36.191592 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17285353-9f7e-4969-8a7d-8de8b7629737 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:36.196871 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 343/660] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:36.197777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c639c9-80cd-477d-97e7-36341be73a0f None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:36.234279 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda386b0-c6b9-4e58-8660-a66df0300742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:36.235238 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:36.236814 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda386b0-c6b9-4e58-8660-a66df0300742 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:36.237572 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda386b0-c6b9-4e58-8660-a66df0300742 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:36.238053 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda386b0-c6b9-4e58-8660-a66df0300742 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:36.308914 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c639c9-80cd-477d-97e7-36341be73a0f None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:36.336659 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda386b0-c6b9-4e58-8660-a66df0300742 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:36.339542 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda386b0-c6b9-4e58-8660-a66df0300742 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:36.345227 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf75dd6-7092-4ca3-9a39-f30ad57525ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:36.345907 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf75dd6-7092-4ca3-9a39-f30ad57525ce None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:36.346190 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf75dd6-7092-4ca3-9a39-f30ad57525ce None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:36.346460 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf75dd6-7092-4ca3-9a39-f30ad57525ce None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:36.347323 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf75dd6-7092-4ca3-9a39-f30ad57525ce None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:36.347624 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf75dd6-7092-4ca3-9a39-f30ad57525ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:36.349031 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf75dd6-7092-4ca3-9a39-f30ad57525ce None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:36.371011 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 344/661] 10.222.0.42 () {56 vars in 1139 bytes} [Mon Sep 12 14:00:36 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:36.371011 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf75dd6-7092-4ca3-9a39-f30ad57525ce None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:36.434991 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:36.452346 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 318/662] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:00:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 14:00:36.453661 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede2041e-67ee-4f0f-b133-6dace3452223 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:36.453948 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede2041e-67ee-4f0f-b133-6dace3452223 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:36.454209 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede2041e-67ee-4f0f-b133-6dace3452223 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:36.455667 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 319/663] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:36.459910 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:36.460881 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede2041e-67ee-4f0f-b133-6dace3452223 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:36.484572 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf75dd6-7092-4ca3-9a39-f30ad57525ce None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:36.520591 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb670e-7245-40a3-a844-e003270979bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:36.521480 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb670e-7245-40a3-a844-e003270979bb None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:36.521756 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb670e-7245-40a3-a844-e003270979bb None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:36.522026 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb670e-7245-40a3-a844-e003270979bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:36.529761 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb670e-7245-40a3-a844-e003270979bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:36.531203 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb670e-7245-40a3-a844-e003270979bb None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:36.544949 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-b4fb670e-7245-40a3-a844-e003270979bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:36.544949 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:36.544949 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:36.544949 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:36.544949 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:36.544949 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:36.544949 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:36.544949 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:36.544949 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:36.544949 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:36.544949 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:36.544949 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:36.544949 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:00:36.544949 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:00:36.544949 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:00:36.545821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:00:36.545821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:36.545821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:36.545821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:36.545821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:36.545821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:36.545821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:36.545821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:36.545821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:36.545821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:36.545821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:36.545821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:36.545821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:36.545821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:36.545821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:36.545821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:36.545821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:36.546701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:00:36.546701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:00:36.546701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:00:36.546701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:00:36.546701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:00:36.546701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:00:36.546701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:36.546701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:36.546701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 345/664] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:00:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:36.548236 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb670e-7245-40a3-a844-e003270979bb None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:36.569383 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede2041e-67ee-4f0f-b133-6dace3452223 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:36.603765 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bee998-00a0-469d-866f-fe8747e07e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:36.604368 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bee998-00a0-469d-866f-fe8747e07e73 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:36.604644 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bee998-00a0-469d-866f-fe8747e07e73 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:36.604912 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bee998-00a0-469d-866f-fe8747e07e73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:36.657547 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb670e-7245-40a3-a844-e003270979bb None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:36.659947 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:36.676034 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bee998-00a0-469d-866f-fe8747e07e73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:36.678172 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bee998-00a0-469d-866f-fe8747e07e73 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:36.703506 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e16334d-4d1c-4ea2-90c5-470ce6a92641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:36.704135 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16334d-4d1c-4ea2-90c5-470ce6a92641 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:36.704519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16334d-4d1c-4ea2-90c5-470ce6a92641 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:36.707054 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16334d-4d1c-4ea2-90c5-470ce6a92641 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:36.710262 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:36.720158 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e16334d-4d1c-4ea2-90c5-470ce6a92641 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:36.721583 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e16334d-4d1c-4ea2-90c5-470ce6a92641 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:36.736613 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-8e16334d-4d1c-4ea2-90c5-470ce6a92641 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:36.736613 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:36.736613 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:36.736613 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:36.736613 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:36.736613 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:36.736613 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:36.736613 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:36.736613 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:36.736613 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:36.736613 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:36.736613 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:36.736613 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:36.736613 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:36.736613 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:36.736613 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:36.737610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:36.737610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:36.737610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:36.737610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:36.737610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:36.737610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:36.737610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:36.737610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:36.737610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:36.737610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:36.737610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:36.737610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:36.737610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:36.737610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:36.737610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:36.737610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:36.737610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:36.738504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:36.738504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:36.738504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:36.738504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 346/665] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:36.739328 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ea4e5-c804-44f6-9a45-730584c69066 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:36.739429 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ea4e5-c804-44f6-9a45-730584c69066 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:36.739683 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ea4e5-c804-44f6-9a45-730584c69066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:36.757901 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-f20ea4e5-c804-44f6-9a45-730584c69066 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:36.759629 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f20ea4e5-c804-44f6-9a45-730584c69066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:36.791831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 320/666] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:00:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 14:00:36.796594 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bee998-00a0-469d-866f-fe8747e07e73 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:36.839253 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 347/667] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:36.839253 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ea4e5-c804-44f6-9a45-730584c69066 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:36.894672 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:36.911385 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bee998-00a0-469d-866f-fe8747e07e73 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:36.947445 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f7b0d-0405-4842-b0ab-3c2a72db9974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:36.948137 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f7b0d-0405-4842-b0ab-3c2a72db9974 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:36.948480 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f7b0d-0405-4842-b0ab-3c2a72db9974 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:36.948818 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f7b0d-0405-4842-b0ab-3c2a72db9974 None admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:36.949581 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71f7b0d-0405-4842-b0ab-3c2a72db9974 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:36.949957 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71f7b0d-0405-4842-b0ab-3c2a72db9974 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:36.951488 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71f7b0d-0405-4842-b0ab-3c2a72db9974 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:36.967196 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 321/668] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 12 14:00:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:36.968149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f7b0d-0405-4842-b0ab-3c2a72db9974 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:36.969009 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ea4e5-c804-44f6-9a45-730584c69066 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:36.991802 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:37.014159 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af889ac5-f164-4725-989b-c7c79512fbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:37.014898 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af889ac5-f164-4725-989b-c7c79512fbe8 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:37.015183 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af889ac5-f164-4725-989b-c7c79512fbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:37.015460 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af889ac5-f164-4725-989b-c7c79512fbe8 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:37.016157 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af889ac5-f164-4725-989b-c7c79512fbe8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:37.016462 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af889ac5-f164-4725-989b-c7c79512fbe8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:37.017810 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af889ac5-f164-4725-989b-c7c79512fbe8 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:37.030382 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 348/669] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:37.031734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af889ac5-f164-4725-989b-c7c79512fbe8 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:37.097158 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f7b0d-0405-4842-b0ab-3c2a72db9974 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:37.131580 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c0abe0-016b-405a-a129-5eafc43a8de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:37.132597 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0abe0-016b-405a-a129-5eafc43a8de8 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:37.132880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0abe0-016b-405a-a129-5eafc43a8de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:37.133957 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0abe0-016b-405a-a129-5eafc43a8de8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:37.145106 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c0abe0-016b-405a-a129-5eafc43a8de8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:37.146553 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c0abe0-016b-405a-a129-5eafc43a8de8 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:37.153480 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:37.158508 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af889ac5-f164-4725-989b-c7c79512fbe8 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:37.162029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-b0c0abe0-016b-405a-a129-5eafc43a8de8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:37.162029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:37.162029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:37.162029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:37.162029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:37.162029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:37.162029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:37.162029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:37.162029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:37.162029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:37.162029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:37.162029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:37.162029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:37.162029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:37.162029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:37.162029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:37.163163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:37.163163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:37.163163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:37.163163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:37.163163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:37.163163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:37.163163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:37.163163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:37.163163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:37.163163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:37.163163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:37.163163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:37.163163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:37.163163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:37.163163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:37.163163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:37.163163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:37.164959 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:37.164959 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:37.164959 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:37.166454 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 322/670] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:37.167455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c0abe0-016b-405a-a129-5eafc43a8de8 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:37.202263 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619da7f-674e-4f95-ad57-0c35cad51a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:37.202900 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619da7f-674e-4f95-ad57-0c35cad51a43 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:37.203184 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619da7f-674e-4f95-ad57-0c35cad51a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:37.203458 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619da7f-674e-4f95-ad57-0c35cad51a43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:37.204189 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e619da7f-674e-4f95-ad57-0c35cad51a43 None admin] RBAC: Adding query filter params (role.id=1c413f4e78ef45b7a821bb884f9cea07, scope.project.id=f666a617f8d14eb18bbeb076032b82f2, user.id=2092a9639067461c9a837a891f56889b) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:37.204494 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e619da7f-674e-4f95-ad57-0c35cad51a43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:37.205907 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e619da7f-674e-4f95-ad57-0c35cad51a43 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:37.211685 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 349/671] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 14:00:37 2022] GET /identity/v3/role_assignments?user.id=2092a9639067461c9a837a891f56889b&scope.project.id=f666a617f8d14eb18bbeb076032b82f2&role.id=1c413f4e78ef45b7a821bb884f9cea07 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:37.212460 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619da7f-674e-4f95-ad57-0c35cad51a43 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:37.272791 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:37.276474 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c0abe0-016b-405a-a129-5eafc43a8de8 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:37.315893 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e3b381-beea-4f9f-b908-4c7ec29b13e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:37.316926 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e3b381-beea-4f9f-b908-4c7ec29b13e9 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:37.317264 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e3b381-beea-4f9f-b908-4c7ec29b13e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:37.317535 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e3b381-beea-4f9f-b908-4c7ec29b13e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:37.325112 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e3b381-beea-4f9f-b908-4c7ec29b13e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:37.326564 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e3b381-beea-4f9f-b908-4c7ec29b13e9 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:37.338530 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619da7f-674e-4f95-ad57-0c35cad51a43 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:37.338788 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-47e3b381-beea-4f9f-b908-4c7ec29b13e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:37.338788 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:37.338788 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:37.338788 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:37.338788 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:37.338788 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:37.338788 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:37.338788 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:37.338788 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:37.338788 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:37.338788 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:37.338788 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:37.338788 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:00:37.338788 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:00:37.338788 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:00:37.339647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:00:37.339647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:37.339647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:37.339647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:37.339647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:37.339647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:37.339647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:37.339647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:37.339647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:37.339647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:37.339647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:37.339647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:37.339647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:37.339647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:37.339647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:37.339647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:37.339647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:37.345977 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:00:37.345977 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:00:37.345977 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:00:37.345977 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:00:37.345977 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:00:37.345977 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:00:37.345977 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:37.345977 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:37.345977 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 323/672] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:00:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:37.345977 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e3b381-beea-4f9f-b908-4c7ec29b13e9 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:37.386439 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e90111-bf40-472b-9897-650542d50896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:37.387028 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e90111-bf40-472b-9897-650542d50896 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:37.387305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e90111-bf40-472b-9897-650542d50896 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:37.387575 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e90111-bf40-472b-9897-650542d50896 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:37.402433 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e90111-bf40-472b-9897-650542d50896 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:37.409527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-60e90111-bf40-472b-9897-650542d50896 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:00:37.409527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:37.409527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:37.409527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:37.409527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:37.409527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:37.409527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:37.409527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:37.409527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:37.409527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:37.409527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:37.409527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:37.409527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:37.409527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:37.409527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:37.409527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:37.410556 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:37.410556 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:37.410556 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:37.410556 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:37.410556 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:37.410556 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:37.410556 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:37.410556 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:37.410556 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:37.410556 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:37.410556 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:37.410556 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:37.410556 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:37.410556 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:37.410556 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:37.410556 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:37.411415 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:37.411415 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:37.411415 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:37.411415 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:37.411415 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:37.411415 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:37.411415 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:37.411415 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:37.411415 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:37.411415 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:37.411415 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:00:37.411415 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:37.412037 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 350/673] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 12 14:00:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:37.413216 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921af7b7-7704-4938-80b0-d46e581649d4 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:37.413493 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921af7b7-7704-4938-80b0-d46e581649d4 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:37.413761 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921af7b7-7704-4938-80b0-d46e581649d4 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:37.415665 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 351/674] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:37.415771 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921af7b7-7704-4938-80b0-d46e581649d4 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:37.425351 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:37.475052 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e3b381-beea-4f9f-b908-4c7ec29b13e9 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:37.509567 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33c9b4-0123-4db5-aa15-c239fb862e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:37.510493 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33c9b4-0123-4db5-aa15-c239fb862e47 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:37.510989 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33c9b4-0123-4db5-aa15-c239fb862e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:37.511429 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33c9b4-0123-4db5-aa15-c239fb862e47 None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:37.512254 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac33c9b4-0123-4db5-aa15-c239fb862e47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:37.512721 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac33c9b4-0123-4db5-aa15-c239fb862e47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:37.514247 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac33c9b4-0123-4db5-aa15-c239fb862e47 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:37.526830 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 324/675] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:37.527869 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33c9b4-0123-4db5-aa15-c239fb862e47 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:37.531786 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921af7b7-7704-4938-80b0-d46e581649d4 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:37.533657 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:37.571771 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc5f84-f7fd-4424-b404-af724b501155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:37.572510 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc5f84-f7fd-4424-b404-af724b501155 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:37.572858 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc5f84-f7fd-4424-b404-af724b501155 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:37.573136 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc5f84-f7fd-4424-b404-af724b501155 None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:37.573828 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc5f84-f7fd-4424-b404-af724b501155 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:37.574128 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc5f84-f7fd-4424-b404-af724b501155 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:37.575555 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc5f84-f7fd-4424-b404-af724b501155 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:37.586982 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 352/676] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 12 14:00:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:37.588153 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c97fe-30be-4b15-9c11-86ccc32ac103 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:37.588438 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c97fe-30be-4b15-9c11-86ccc32ac103 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:37.588710 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c97fe-30be-4b15-9c11-86ccc32ac103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:37.607554 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-986c97fe-30be-4b15-9c11-86ccc32ac103 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:37.608733 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-986c97fe-30be-4b15-9c11-86ccc32ac103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:37.640944 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33c9b4-0123-4db5-aa15-c239fb862e47 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:37.658630 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:37.676734 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e9d6b-385a-466e-bd97-636240658e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:37.677398 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e9d6b-385a-466e-bd97-636240658e54 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:37.677603 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e9d6b-385a-466e-bd97-636240658e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:37.677873 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e9d6b-385a-466e-bd97-636240658e54 None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:37.678520 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16e9d6b-385a-466e-bd97-636240658e54 None admin] RBAC: Adding query filter params (name=service) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:37.678899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16e9d6b-385a-466e-bd97-636240658e54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:37.680763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16e9d6b-385a-466e-bd97-636240658e54 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:37.690960 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 353/677] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:37.690960 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986c97fe-30be-4b15-9c11-86ccc32ac103 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:37.691201 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 325/678] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 12 14:00:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:37.691201 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e9d6b-385a-466e-bd97-636240658e54 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:37.800879 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:37.812828 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986c97fe-30be-4b15-9c11-86ccc32ac103 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:37.836165 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e9d6b-385a-466e-bd97-636240658e54 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:37.862732 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62902854-85e9-478f-b808-23dcc5745ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:37.863866 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62902854-85e9-478f-b808-23dcc5745ba3 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:37.864413 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62902854-85e9-478f-b808-23dcc5745ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:37.864898 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62902854-85e9-478f-b808-23dcc5745ba3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:37.872374 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5465d92-10cf-48ee-96d1-e40f83298b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:37.872978 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5465d92-10cf-48ee-96d1-e40f83298b67 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:37.873276 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5465d92-10cf-48ee-96d1-e40f83298b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:37.873505 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5465d92-10cf-48ee-96d1-e40f83298b67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:37.874216 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5465d92-10cf-48ee-96d1-e40f83298b67 None admin] RBAC: Adding query filter params (role.id=1c413f4e78ef45b7a821bb884f9cea07, scope.project.id=f666a617f8d14eb18bbeb076032b82f2, user.id=8da38bc8dd2f4cf798649ce96097a4b8) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:37.874505 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62902854-85e9-478f-b808-23dcc5745ba3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:37.874609 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5465d92-10cf-48ee-96d1-e40f83298b67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:37.875952 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5465d92-10cf-48ee-96d1-e40f83298b67 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:37.876431 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62902854-85e9-478f-b808-23dcc5745ba3 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:37.882165 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 326/679] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 14:00:37 2022] GET /identity/v3/role_assignments?user.id=8da38bc8dd2f4cf798649ce96097a4b8&scope.project.id=f666a617f8d14eb18bbeb076032b82f2&role.id=1c413f4e78ef45b7a821bb884f9cea07 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:37.883633 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5465d92-10cf-48ee-96d1-e40f83298b67 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:37.890699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-62902854-85e9-478f-b808-23dcc5745ba3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:37.890699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:37.890699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:37.890699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:37.890699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:37.890699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:37.890699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:37.890699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:37.890699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:37.890699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:37.890699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:37.890699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:37.890699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:00:37.890699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:00:37.890699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:00:37.891749 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:00:37.891749 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:37.891749 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:37.891749 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:37.891749 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:37.891749 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:37.891749 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:37.891749 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:37.891749 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:37.891749 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:37.891749 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:37.891749 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:37.891749 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:37.891749 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:37.891749 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:37.891749 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:37.891749 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:37.892666 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:00:37.892666 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:00:37.892666 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:00:37.892666 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:00:37.892666 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:00:37.892666 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:00:37.892666 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:37.892666 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:37.896022 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 354/680] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:00:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:37.906432 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62902854-85e9-478f-b808-23dcc5745ba3 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:37.974809 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:38.043162 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5465d92-10cf-48ee-96d1-e40f83298b67 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:38.075934 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62902854-85e9-478f-b808-23dcc5745ba3 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:38.081842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b845047-fc0c-4b27-8ab6-98e615d82dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:38.082781 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b845047-fc0c-4b27-8ab6-98e615d82dad None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:38.083285 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b845047-fc0c-4b27-8ab6-98e615d82dad None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:38.083751 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b845047-fc0c-4b27-8ab6-98e615d82dad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:38.098896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b845047-fc0c-4b27-8ab6-98e615d82dad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:38.100727 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b845047-fc0c-4b27-8ab6-98e615d82dad None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:38.116593 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750747a1-0caf-4f2e-9bf2-592c3b4d74f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:38.117203 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750747a1-0caf-4f2e-9bf2-592c3b4d74f9 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:38.117515 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750747a1-0caf-4f2e-9bf2-592c3b4d74f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:38.117790 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750747a1-0caf-4f2e-9bf2-592c3b4d74f9 None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:38.118488 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750747a1-0caf-4f2e-9bf2-592c3b4d74f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:38.118796 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750747a1-0caf-4f2e-9bf2-592c3b4d74f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:38.119386 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-8b845047-fc0c-4b27-8ab6-98e615d82dad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:38.119386 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:38.119386 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:38.119386 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:38.119386 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:38.119386 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:38.119386 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:38.119386 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:38.119386 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:38.119386 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:38.119386 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:38.119386 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:38.119386 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:38.119386 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:38.119386 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:38.119386 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:38.120278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:38.120278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:38.120278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:38.120278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:38.120278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:38.120278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:38.120278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:38.120278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:38.120278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:38.120278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:38.120278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:38.120278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:38.120278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:38.120278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:38.120278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:38.120278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:38.120278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:38.121164 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:38.121164 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:38.121164 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:38.121332 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750747a1-0caf-4f2e-9bf2-592c3b4d74f9 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:38.123288 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 327/681] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:38.124273 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b845047-fc0c-4b27-8ab6-98e615d82dad None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:38.134579 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 355/682] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 12 14:00:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:38.135235 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750747a1-0caf-4f2e-9bf2-592c3b4d74f9 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:38.152704 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:38.248210 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750747a1-0caf-4f2e-9bf2-592c3b4d74f9 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:38.259676 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:38.268841 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b845047-fc0c-4b27-8ab6-98e615d82dad None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:38.285780 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea59986c-082b-4fb3-b538-0a5eb7b52025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:38.286486 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea59986c-082b-4fb3-b538-0a5eb7b52025 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:38.286775 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea59986c-082b-4fb3-b538-0a5eb7b52025 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:38.287149 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea59986c-082b-4fb3-b538-0a5eb7b52025 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:38.287830 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea59986c-082b-4fb3-b538-0a5eb7b52025 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:38.288409 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea59986c-082b-4fb3-b538-0a5eb7b52025 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:38.289651 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea59986c-082b-4fb3-b538-0a5eb7b52025 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:38.309399 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 356/683] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:38.310554 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea59986c-082b-4fb3-b538-0a5eb7b52025 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:38.317932 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65e613-c5fd-4a85-83ce-2155d8d7006d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:38.318956 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65e613-c5fd-4a85-83ce-2155d8d7006d None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:38.319450 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65e613-c5fd-4a85-83ce-2155d8d7006d None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:38.319923 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65e613-c5fd-4a85-83ce-2155d8d7006d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:38.332881 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc65e613-c5fd-4a85-83ce-2155d8d7006d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:38.338794 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-bc65e613-c5fd-4a85-83ce-2155d8d7006d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:00:38.338794 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:38.338794 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:38.338794 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:38.338794 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:38.338794 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:38.338794 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:38.338794 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:38.338794 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:38.338794 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:38.338794 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:38.338794 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:38.338794 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:38.338794 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:38.338794 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:38.338794 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:38.339763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:38.339763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:38.339763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:38.339763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:38.339763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:38.339763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:38.339763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:38.339763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:38.339763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:38.339763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:38.339763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:38.339763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:38.339763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:38.339763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:38.339763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:38.339763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:38.340613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:38.340613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:38.340613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:38.340613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:38.340613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:38.340613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:38.340613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:38.340613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:38.340613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:38.340613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:38.340613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:00:38.340613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:38.343307 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 328/684] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 12 14:00:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:38.344456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65e613-c5fd-4a85-83ce-2155d8d7006d None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:38.395598 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:38.431427 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea59986c-082b-4fb3-b538-0a5eb7b52025 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:38.451827 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65e613-c5fd-4a85-83ce-2155d8d7006d None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:38.472017 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86a2ca-96da-40c8-8731-5da2d37546d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:38.472637 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86a2ca-96da-40c8-8731-5da2d37546d4 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:38.473011 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86a2ca-96da-40c8-8731-5da2d37546d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:38.473342 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86a2ca-96da-40c8-8731-5da2d37546d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:38.478613 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa86a2ca-96da-40c8-8731-5da2d37546d4 None admin] RBAC: Adding query filter params (role.id=1c413f4e78ef45b7a821bb884f9cea07, scope.project.id=f666a617f8d14eb18bbeb076032b82f2, user.id=865b1647468c4efe8efe4febe2203a58) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:38.478613 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa86a2ca-96da-40c8-8731-5da2d37546d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:38.478613 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa86a2ca-96da-40c8-8731-5da2d37546d4 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:38.488062 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 357/685] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 14:00:38 2022] GET /identity/v3/role_assignments?user.id=865b1647468c4efe8efe4febe2203a58&scope.project.id=f666a617f8d14eb18bbeb076032b82f2&role.id=1c413f4e78ef45b7a821bb884f9cea07 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:38.488062 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86a2ca-96da-40c8-8731-5da2d37546d4 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:38.510163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23ef21-8da7-4d6b-beca-7e5b0ceedf44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:38.510805 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23ef21-8da7-4d6b-beca-7e5b0ceedf44 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:38.511248 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23ef21-8da7-4d6b-beca-7e5b0ceedf44 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:38.511526 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23ef21-8da7-4d6b-beca-7e5b0ceedf44 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:38.528964 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd23ef21-8da7-4d6b-beca-7e5b0ceedf44 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:38.530459 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd23ef21-8da7-4d6b-beca-7e5b0ceedf44 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:38.558125 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-bd23ef21-8da7-4d6b-beca-7e5b0ceedf44 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 12 14:00:38.558125 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:38.558125 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:38.558125 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:38.558125 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:38.558125 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:38.558125 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:38.558125 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:38.558125 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:38.558125 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:38.558125 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:38.558125 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:38.558125 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:00:38.558125 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:00:38.558125 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:00:38.558125 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:00:38.559213 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:38.559213 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:38.559213 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:00:38.559213 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:38.559213 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:00:38.559213 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:38.559213 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:38.559213 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:38.559213 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:38.559213 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:38.559213 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:38.559213 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:38.559213 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:38.559213 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:38.559213 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:38.559213 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:38.559213 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:38.559213 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:38.560589 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:38.560589 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:38.560589 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:00:38.560589 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:00:38.560589 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:00:38.560589 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:00:38.560589 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:00:38.560589 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:00:38.560589 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 12 14:00:38.560589 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:38.560589 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 329/686] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 12 14:00:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:38.567511 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23ef21-8da7-4d6b-beca-7e5b0ceedf44 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:38.606181 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:38.679414 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86a2ca-96da-40c8-8731-5da2d37546d4 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:38.731034 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc0b67-d4c0-4f6f-a167-9d3fdec0abdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:38.731653 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc0b67-d4c0-4f6f-a167-9d3fdec0abdf None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:38.731929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc0b67-d4c0-4f6f-a167-9d3fdec0abdf None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:38.732312 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc0b67-d4c0-4f6f-a167-9d3fdec0abdf None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:38.732978 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacc0b67-d4c0-4f6f-a167-9d3fdec0abdf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:38.733277 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacc0b67-d4c0-4f6f-a167-9d3fdec0abdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:38.734639 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacc0b67-d4c0-4f6f-a167-9d3fdec0abdf None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:38.738463 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:38.753870 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 358/687] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 14:00:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:38.755052 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc0b67-d4c0-4f6f-a167-9d3fdec0abdf None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:38.759657 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23ef21-8da7-4d6b-beca-7e5b0ceedf44 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:38.798669 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41180fc4-b434-4955-931b-cdb629ca7d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:38.799309 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41180fc4-b434-4955-931b-cdb629ca7d94 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:38.799584 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41180fc4-b434-4955-931b-cdb629ca7d94 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:38.799852 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41180fc4-b434-4955-931b-cdb629ca7d94 None admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:38.800791 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41180fc4-b434-4955-931b-cdb629ca7d94 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:38.801053 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41180fc4-b434-4955-931b-cdb629ca7d94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:38.802552 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41180fc4-b434-4955-931b-cdb629ca7d94 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:38.825957 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 330/688] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Sep 12 14:00:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:38.838149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41180fc4-b434-4955-931b-cdb629ca7d94 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:38.870084 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc0b67-d4c0-4f6f-a167-9d3fdec0abdf None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:38.930114 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f80a4-4888-4590-ac08-24cd987ce74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:38.930749 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f80a4-4888-4590-ac08-24cd987ce74b None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:38.931058 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f80a4-4888-4590-ac08-24cd987ce74b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:38.931331 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f80a4-4888-4590-ac08-24cd987ce74b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:38.946797 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9f80a4-4888-4590-ac08-24cd987ce74b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:38.948258 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9f80a4-4888-4590-ac08-24cd987ce74b None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:38.973200 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:38.982054 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-fa9f80a4-4888-4590-ac08-24cd987ce74b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:38.982054 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:38.982054 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:38.982054 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:38.982054 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:38.982054 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:38.982054 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:38.982054 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:38.982054 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:38.982054 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:38.982054 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:38.982054 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:38.982054 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:38.982054 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:38.982054 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:38.982054 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:38.983097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:38.983097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:38.983097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:38.983097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:38.983097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:38.983097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:38.983097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:38.983097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:38.983097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:38.983097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:38.983097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:38.983097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:38.983097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:38.983097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:38.983097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:38.983097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:38.983097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:38.984037 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:38.984037 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:38.984037 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:38.984618 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 359/689] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:38.991819 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f80a4-4888-4590-ac08-24cd987ce74b None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:39.033684 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41180fc4-b434-4955-931b-cdb629ca7d94 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:39.077414 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:39.077931 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a210aff5-f9d1-4e1c-a7f8-67f891d27006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:39.078201 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210aff5-f9d1-4e1c-a7f8-67f891d27006 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:39.078398 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210aff5-f9d1-4e1c-a7f8-67f891d27006 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:39.078617 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210aff5-f9d1-4e1c-a7f8-67f891d27006 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:39.173112 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f80a4-4888-4590-ac08-24cd987ce74b None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:39.207905 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a210aff5-f9d1-4e1c-a7f8-67f891d27006 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:39.209369 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a210aff5-f9d1-4e1c-a7f8-67f891d27006 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:39.229475 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc356d8a-dd4d-420b-8fff-9d61ff47cb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:39.230086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc356d8a-dd4d-420b-8fff-9d61ff47cb24 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:39.230367 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc356d8a-dd4d-420b-8fff-9d61ff47cb24 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:39.230641 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc356d8a-dd4d-420b-8fff-9d61ff47cb24 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:39.231326 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc356d8a-dd4d-420b-8fff-9d61ff47cb24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:39.231627 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc356d8a-dd4d-420b-8fff-9d61ff47cb24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:39.233036 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc356d8a-dd4d-420b-8fff-9d61ff47cb24 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:39.263697 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 360/690] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:39.267112 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc356d8a-dd4d-420b-8fff-9d61ff47cb24 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:39.347196 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:39.401577 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc356d8a-dd4d-420b-8fff-9d61ff47cb24 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:39.407823 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 331/691] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:00:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 14:00:39.413552 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:39.413773 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a210aff5-f9d1-4e1c-a7f8-67f891d27006 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:39.442483 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dfaca6-0a5c-4f41-81bc-1e6383ff5a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:39.443397 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfaca6-0a5c-4f41-81bc-1e6383ff5a30 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:39.444012 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfaca6-0a5c-4f41-81bc-1e6383ff5a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:39.444433 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfaca6-0a5c-4f41-81bc-1e6383ff5a30 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:39.457550 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dfaca6-0a5c-4f41-81bc-1e6383ff5a30 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:39.459186 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dfaca6-0a5c-4f41-81bc-1e6383ff5a30 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:39.484109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-f0dfaca6-0a5c-4f41-81bc-1e6383ff5a30 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 14:00:39.484109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:39.484109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:39.484109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:39.484109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:39.484109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:39.484109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:39.484109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:39.484109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:39.484109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:39.484109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:39.484109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:39.484109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:00:39.484109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:00:39.484109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:00:39.484109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:00:39.485373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:39.485373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:39.485373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:00:39.485373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:39.485373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:00:39.485373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:39.485373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:39.485373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:39.485373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:39.485373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:39.485373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:39.485373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:39.485373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:39.485373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:39.485373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:39.485373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:39.485373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:39.485373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:39.486620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:39.486620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:39.486620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:00:39.486620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:00:39.486620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:00:39.486620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:00:39.486620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:00:39.486620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:00:39.486620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 14:00:39.486620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:39.489608 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 361/692] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Sep 12 14:00:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:00:39.497037 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dfaca6-0a5c-4f41-81bc-1e6383ff5a30 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:39.626479 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a210aff5-f9d1-4e1c-a7f8-67f891d27006 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:39.660325 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f607dbc-f9b1-4db7-a70d-02be26b71571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:39.661155 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f607dbc-f9b1-4db7-a70d-02be26b71571 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:39.661277 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f607dbc-f9b1-4db7-a70d-02be26b71571 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:39.661477 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f607dbc-f9b1-4db7-a70d-02be26b71571 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:39.669566 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dfaca6-0a5c-4f41-81bc-1e6383ff5a30 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:39.672027 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f607dbc-f9b1-4db7-a70d-02be26b71571 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:39.673873 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f607dbc-f9b1-4db7-a70d-02be26b71571 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:39.688617 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-9f607dbc-f9b1-4db7-a70d-02be26b71571 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:39.688617 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:39.688617 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:39.688617 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:39.688617 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:39.688617 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:39.688617 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:39.688617 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:39.688617 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:39.688617 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:39.688617 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:39.688617 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:39.688617 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:39.688617 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:39.688617 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:39.688617 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:39.689778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:39.689778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:39.689778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:39.689778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:39.689778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:39.689778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:39.689778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:39.689778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:39.689778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:39.689778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:39.689778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:39.689778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:39.689778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:39.689778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:39.689778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:39.689778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:39.689778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:39.690781 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:39.690781 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:39.690781 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:39.690781 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 332/693] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:00:39.691226 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a0723-f6c0-4df2-8877-90cc7ef4e898 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:39.691556 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a0723-f6c0-4df2-8877-90cc7ef4e898 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:39.691829 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a0723-f6c0-4df2-8877-90cc7ef4e898 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:39.697486 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 333/694] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:39.698274 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a0723-f6c0-4df2-8877-90cc7ef4e898 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:39.717046 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4082b8-7809-4d51-b2bd-332404732873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:39.717730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4082b8-7809-4d51-b2bd-332404732873 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:39.718008 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4082b8-7809-4d51-b2bd-332404732873 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:39.718275 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4082b8-7809-4d51-b2bd-332404732873 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:39.719076 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4082b8-7809-4d51-b2bd-332404732873 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:39.719476 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4082b8-7809-4d51-b2bd-332404732873 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:39.721060 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4082b8-7809-4d51-b2bd-332404732873 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:39.732739 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:39.747418 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 362/695] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Sep 12 14:00:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:39.748667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1e171-3a4a-4047-ab90-ac0fa251b070 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:39.748953 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1e171-3a4a-4047-ab90-ac0fa251b070 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:39.749224 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1e171-3a4a-4047-ab90-ac0fa251b070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:39.768780 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-cde1e171-3a4a-4047-ab90-ac0fa251b070 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:39.770076 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-cde1e171-3a4a-4047-ab90-ac0fa251b070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:39.795066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:39.832444 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a0723-f6c0-4df2-8877-90cc7ef4e898 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:39.856334 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 363/696] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:39.857521 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde1e171-3a4a-4047-ab90-ac0fa251b070 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:39.901975 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8cd59-ade2-4419-8d46-b8c50213ae72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:39.902588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8cd59-ade2-4419-8d46-b8c50213ae72 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:39.902882 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8cd59-ade2-4419-8d46-b8c50213ae72 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:39.903157 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8cd59-ade2-4419-8d46-b8c50213ae72 None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:39.903797 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d8cd59-ade2-4419-8d46-b8c50213ae72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:39.904094 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d8cd59-ade2-4419-8d46-b8c50213ae72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:39.905452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d8cd59-ade2-4419-8d46-b8c50213ae72 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:39.934055 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 334/697] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:39.939239 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8cd59-ade2-4419-8d46-b8c50213ae72 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:39.987688 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde1e171-3a4a-4047-ab90-ac0fa251b070 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:40.044622 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:40.061962 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8cd59-ade2-4419-8d46-b8c50213ae72 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:40.072291 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e83b9-2228-48db-a9a9-2a54d7e17d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:40.073002 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e83b9-2228-48db-a9a9-2a54d7e17d8b None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:40.073180 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e83b9-2228-48db-a9a9-2a54d7e17d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:40.073438 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e83b9-2228-48db-a9a9-2a54d7e17d8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:40.106891 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748fc25-913b-4823-87da-fa2ef3c7c7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:40.108023 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748fc25-913b-4823-87da-fa2ef3c7c7c3 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:40.108526 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748fc25-913b-4823-87da-fa2ef3c7c7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:40.108989 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748fc25-913b-4823-87da-fa2ef3c7c7c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:40.120559 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:40.124404 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c748fc25-913b-4823-87da-fa2ef3c7c7c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:40.126060 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c748fc25-913b-4823-87da-fa2ef3c7c7c3 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:40.152301 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-c748fc25-913b-4823-87da-fa2ef3c7c7c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:40.152301 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:40.152301 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:40.152301 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:40.152301 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:40.152301 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:40.152301 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:40.152301 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:40.152301 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:40.152301 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:40.152301 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:40.152301 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:40.152301 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:00:40.152301 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:00:40.152301 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:00:40.153211 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:00:40.153211 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:40.153211 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:40.153211 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:40.153211 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:40.153211 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:40.153211 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:40.153211 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:40.153211 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:40.153211 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:40.153211 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:40.153211 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:40.153211 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:40.153211 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:40.153211 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:40.153211 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:40.153211 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:40.154136 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:00:40.154136 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:00:40.154136 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:00:40.154136 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:00:40.154136 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:00:40.154136 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:00:40.154136 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:40.154136 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:40.156452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 335/698] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:00:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:40.161906 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748fc25-913b-4823-87da-fa2ef3c7c7c3 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:40.162357 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e83b9-2228-48db-a9a9-2a54d7e17d8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:40.163812 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e83b9-2228-48db-a9a9-2a54d7e17d8b None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:40.274337 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748fc25-913b-4823-87da-fa2ef3c7c7c3 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:40.275697 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 364/699] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:00:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 14:00:40.276952 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e83b9-2228-48db-a9a9-2a54d7e17d8b None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:40.318132 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e7cd0-3d0d-480d-9f60-bc9f2fa7906d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:40.318756 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e7cd0-3d0d-480d-9f60-bc9f2fa7906d None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:40.319121 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e7cd0-3d0d-480d-9f60-bc9f2fa7906d None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:40.319455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e7cd0-3d0d-480d-9f60-bc9f2fa7906d None admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:40.320211 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e7cd0-3d0d-480d-9f60-bc9f2fa7906d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:40.320605 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e7cd0-3d0d-480d-9f60-bc9f2fa7906d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:40.323472 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e7cd0-3d0d-480d-9f60-bc9f2fa7906d None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:40.339789 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 336/700] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 12 14:00:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:40.340642 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:40.341629 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e7cd0-3d0d-480d-9f60-bc9f2fa7906d None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:40.356178 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:40.393667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e83b9-2228-48db-a9a9-2a54d7e17d8b None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:40.428308 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab8e898-41c4-4f0e-9465-7c63db1d0145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:40.429022 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8e898-41c4-4f0e-9465-7c63db1d0145 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:40.429305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8e898-41c4-4f0e-9465-7c63db1d0145 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:40.429569 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8e898-41c4-4f0e-9465-7c63db1d0145 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:40.439215 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab8e898-41c4-4f0e-9465-7c63db1d0145 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:40.440633 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab8e898-41c4-4f0e-9465-7c63db1d0145 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:40.455468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-cab8e898-41c4-4f0e-9465-7c63db1d0145 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:40.455468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:40.455468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:40.455468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:40.455468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:40.455468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:40.455468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:40.455468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:40.455468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:40.455468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:40.455468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:40.455468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:40.455468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:40.455468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:40.455468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:40.455468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:40.456445 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:40.456445 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:40.456445 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:40.456445 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:40.456445 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:40.456445 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:40.456445 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:40.456445 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:40.456445 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:40.456445 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:40.456445 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:40.456445 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:40.456445 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:40.456445 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:40.456445 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:40.456445 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:40.456445 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:40.457357 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:40.457357 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:40.457357 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:40.457357 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 365/701] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:40.457357 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab8e898-41c4-4f0e-9465-7c63db1d0145 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:40.470224 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e7cd0-3d0d-480d-9f60-bc9f2fa7906d None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:40.504385 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1caf8e-eb34-48fd-ab70-3cf8e5afcabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:40.505053 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1caf8e-eb34-48fd-ab70-3cf8e5afcabe None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:40.505329 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1caf8e-eb34-48fd-ab70-3cf8e5afcabe None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:40.505595 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1caf8e-eb34-48fd-ab70-3cf8e5afcabe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:40.516351 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1caf8e-eb34-48fd-ab70-3cf8e5afcabe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:40.518025 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1caf8e-eb34-48fd-ab70-3cf8e5afcabe None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:40.532882 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-ab1caf8e-eb34-48fd-ab70-3cf8e5afcabe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:40.532882 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:40.532882 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:40.532882 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:40.532882 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:40.532882 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:40.532882 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:40.532882 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:40.532882 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:40.532882 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:40.532882 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:40.532882 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:40.532882 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:40.532882 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:40.532882 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:40.532882 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:40.533821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:40.533821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:40.533821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:40.533821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:40.533821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:40.533821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:40.533821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:40.533821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:40.533821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:40.533821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:40.533821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:40.533821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:40.533821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:40.533821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:40.533821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:40.533821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:40.533821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:40.534694 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:40.534694 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:40.534694 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:40.540528 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 337/702] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:40.541004 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f1706-8810-4805-9ab2-29b6636e122b None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:40.541650 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f1706-8810-4805-9ab2-29b6636e122b None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:40.541927 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f1706-8810-4805-9ab2-29b6636e122b None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:40.543750 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 338/703] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:40.544302 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f1706-8810-4805-9ab2-29b6636e122b None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:40.571886 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab8e898-41c4-4f0e-9465-7c63db1d0145 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:40.582901 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:40.609594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c4b658-c2bd-40ec-8164-a932c14900fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:40.610196 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c4b658-c2bd-40ec-8164-a932c14900fb None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:40.610474 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c4b658-c2bd-40ec-8164-a932c14900fb None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:40.610748 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c4b658-c2bd-40ec-8164-a932c14900fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:40.611499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c4b658-c2bd-40ec-8164-a932c14900fb None admin] RBAC: Adding query filter params (role.id=1c413f4e78ef45b7a821bb884f9cea07, scope.project.id=f666a617f8d14eb18bbeb076032b82f2, user.id=25a87ea9c3c5409b97a35c4857ae3ca6) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:40.611804 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c4b658-c2bd-40ec-8164-a932c14900fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:40.613216 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c4b658-c2bd-40ec-8164-a932c14900fb None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:40.615496 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:40.630879 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 366/704] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 14:00:40 2022] GET /identity/v3/role_assignments?user.id=25a87ea9c3c5409b97a35c4857ae3ca6&scope.project.id=f666a617f8d14eb18bbeb076032b82f2&role.id=1c413f4e78ef45b7a821bb884f9cea07 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:40.630879 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c4b658-c2bd-40ec-8164-a932c14900fb None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:40.679020 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f1706-8810-4805-9ab2-29b6636e122b None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:40.724493 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004179e9-47f3-416e-9885-6d7131819fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:40.725214 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004179e9-47f3-416e-9885-6d7131819fed None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:40.725695 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004179e9-47f3-416e-9885-6d7131819fed None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:40.726023 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004179e9-47f3-416e-9885-6d7131819fed None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:40.727441 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004179e9-47f3-416e-9885-6d7131819fed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:40.727749 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004179e9-47f3-416e-9885-6d7131819fed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:40.729173 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004179e9-47f3-416e-9885-6d7131819fed None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:40.744189 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 339/705] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:40.745539 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e74f79-7a72-4223-b4cc-1d3739586b34 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:40.745820 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e74f79-7a72-4223-b4cc-1d3739586b34 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:40.746121 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e74f79-7a72-4223-b4cc-1d3739586b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:40.767313 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-75e74f79-7a72-4223-b4cc-1d3739586b34 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:40.768511 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-75e74f79-7a72-4223-b4cc-1d3739586b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:40.768975 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c4b658-c2bd-40ec-8164-a932c14900fb None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:40.813251 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747fdf1-0448-42b6-bf65-6cedf6f00aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:40.814665 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747fdf1-0448-42b6-bf65-6cedf6f00aef None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:40.814986 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747fdf1-0448-42b6-bf65-6cedf6f00aef None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:40.815263 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747fdf1-0448-42b6-bf65-6cedf6f00aef None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:40.815912 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2747fdf1-0448-42b6-bf65-6cedf6f00aef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:40.816217 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2747fdf1-0448-42b6-bf65-6cedf6f00aef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:40.819581 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2747fdf1-0448-42b6-bf65-6cedf6f00aef None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:40.833843 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 367/706] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:40.833843 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747fdf1-0448-42b6-bf65-6cedf6f00aef None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:40.863511 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 340/707] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:40.864834 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e74f79-7a72-4223-b4cc-1d3739586b34 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:40.894704 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:40.930986 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:40.975408 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747fdf1-0448-42b6-bf65-6cedf6f00aef None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:41.003920 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e74f79-7a72-4223-b4cc-1d3739586b34 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:41.027127 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eafe30-2688-421a-9954-031a12c79b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:41.028674 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eafe30-2688-421a-9954-031a12c79b2d None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:41.029418 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eafe30-2688-421a-9954-031a12c79b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:41.029906 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eafe30-2688-421a-9954-031a12c79b2d None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:41.031165 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eafe30-2688-421a-9954-031a12c79b2d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:41.033228 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eafe30-2688-421a-9954-031a12c79b2d None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:41.053314 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6925dd-e321-49fb-96df-95f0d381f9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:41.054118 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6925dd-e321-49fb-96df-95f0d381f9eb None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:41.054265 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6925dd-e321-49fb-96df-95f0d381f9eb None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:41.054468 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6925dd-e321-49fb-96df-95f0d381f9eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:41.062002 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6925dd-e321-49fb-96df-95f0d381f9eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:41.063533 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6925dd-e321-49fb-96df-95f0d381f9eb None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:41.064028 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 368/708] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Sep 12 14:00:40 2022] POST /identity/v3/users => generated 254 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:00:41.078879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-ed6925dd-e321-49fb-96df-95f0d381f9eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:41.078879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:41.078879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:41.078879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:41.078879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:41.078879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:41.078879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:41.078879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:41.078879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:41.078879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:41.078879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:41.078879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:41.078879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:00:41.078879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:00:41.078879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:00:41.079856 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:00:41.079856 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:41.079856 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:41.079856 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:41.079856 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:41.079856 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:41.079856 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:41.079856 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:41.079856 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:41.079856 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:41.079856 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:41.079856 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:41.079856 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:41.079856 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:41.079856 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:41.079856 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:41.079856 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:41.080743 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:00:41.080743 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:00:41.080743 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:00:41.080743 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:00:41.080743 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:00:41.080743 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:00:41.080743 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:41.080743 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:41.080743 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 341/709] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:00:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:41.087035 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eafe30-2688-421a-9954-031a12c79b2d None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:41.140417 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52765a45-21dc-4034-b1e0-4ad9379d8b2f None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:41.140750 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52765a45-21dc-4034-b1e0-4ad9379d8b2f None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:41.140961 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52765a45-21dc-4034-b1e0-4ad9379d8b2f None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:41.142297 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 342/710] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 14:00:41.147869 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb541b2b-81ea-4bd8-9434-f17b5ed7aca6 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:41.148178 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb541b2b-81ea-4bd8-9434-f17b5ed7aca6 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:41.148429 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb541b2b-81ea-4bd8-9434-f17b5ed7aca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:41.168831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-cb541b2b-81ea-4bd8-9434-f17b5ed7aca6 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:41.170078 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-cb541b2b-81ea-4bd8-9434-f17b5ed7aca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:41.203251 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:41.209877 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eafe30-2688-421a-9954-031a12c79b2d None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:41.253549 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c910f90-f477-4aa8-b80f-9b468cf2cb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:41.254248 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c910f90-f477-4aa8-b80f-9b468cf2cb4d None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:41.254528 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c910f90-f477-4aa8-b80f-9b468cf2cb4d None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:41.254798 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c910f90-f477-4aa8-b80f-9b468cf2cb4d None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:41.255558 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c910f90-f477-4aa8-b80f-9b468cf2cb4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:41.255862 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c910f90-f477-4aa8-b80f-9b468cf2cb4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:41.257318 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c910f90-f477-4aa8-b80f-9b468cf2cb4d None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:41.261585 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 343/711] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 14:00:41.262470 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb541b2b-81ea-4bd8-9434-f17b5ed7aca6 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:41.275026 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 369/712] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 12 14:00:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:41.281602 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c910f90-f477-4aa8-b80f-9b468cf2cb4d None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:41.323457 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:41.392141 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb541b2b-81ea-4bd8-9434-f17b5ed7aca6 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:41.396308 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c910f90-f477-4aa8-b80f-9b468cf2cb4d None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:41.435934 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f550b4-34c5-4c1d-babe-d4be0d8958b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:41.436191 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8917bbc-a1fd-40f9-9e98-1c22c8b9d5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:41.436546 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8917bbc-a1fd-40f9-9e98-1c22c8b9d5f7 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:41.436825 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8917bbc-a1fd-40f9-9e98-1c22c8b9d5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:41.437095 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8917bbc-a1fd-40f9-9e98-1c22c8b9d5f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:41.437603 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f550b4-34c5-4c1d-babe-d4be0d8958b4 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:41.437832 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8917bbc-a1fd-40f9-9e98-1c22c8b9d5f7 None admin] RBAC: Adding query filter params (role.id=e14564f00daf48a28d4e32da8cd2efad, scope.project.id=f666a617f8d14eb18bbeb076032b82f2, user.id=854ab2dcec5a48b18f409b33cb61b5da) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:41.438134 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8917bbc-a1fd-40f9-9e98-1c22c8b9d5f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:41.438483 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f550b4-34c5-4c1d-babe-d4be0d8958b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:41.439118 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f550b4-34c5-4c1d-babe-d4be0d8958b4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:41.439572 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8917bbc-a1fd-40f9-9e98-1c22c8b9d5f7 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:41.440417 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f550b4-34c5-4c1d-babe-d4be0d8958b4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:41.443326 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f550b4-34c5-4c1d-babe-d4be0d8958b4 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:41.450861 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 370/713] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 14:00:41 2022] GET /identity/v3/role_assignments?user.id=854ab2dcec5a48b18f409b33cb61b5da&scope.project.id=f666a617f8d14eb18bbeb076032b82f2&role.id=e14564f00daf48a28d4e32da8cd2efad => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:00:41.462416 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-17f550b4-34c5-4c1d-babe-d4be0d8958b4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 14:00:41.462416 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:41.462416 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 14:00:41.462416 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 14:00:41.462416 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 14:00:41.462416 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 14:00:41.462416 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 14:00:41.462416 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:00:41.462416 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 14:00:41.462416 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:41.462416 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 14:00:41.462416 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:41.462416 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:41.462416 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:41.462416 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:41.462416 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:41.462416 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:41.463638 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:41.463638 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:41.463638 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:41.463638 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:41.463638 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:41.463638 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:41.463638 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 14:00:41.463638 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 14:00:41.463638 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 14:00:41.463638 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 14:00:41.463638 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:41.463638 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:41.463638 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:41.463638 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:41.463638 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:41.463638 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:41.464489 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:41.464489 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:41.464489 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:41.464489 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:41.464489 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:41.464489 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:41.464489 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:41.464489 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:41.464489 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:41.464489 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:41.464489 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 14:00:41.464489 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:00:41.464489 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 14:00:41.464489 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:41.464489 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 344/714] 10.222.0.42 () {56 vars in 1129 bytes} [Mon Sep 12 14:00:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:41.471553 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8917bbc-a1fd-40f9-9e98-1c22c8b9d5f7 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:41.490457 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:41.584099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f550b4-34c5-4c1d-babe-d4be0d8958b4 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:41.587184 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8917bbc-a1fd-40f9-9e98-1c22c8b9d5f7 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:41.637705 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2da680-12aa-40dd-8ec5-ebebf99916e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:41.638440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2da680-12aa-40dd-8ec5-ebebf99916e5 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:41.638722 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2da680-12aa-40dd-8ec5-ebebf99916e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:41.639017 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2da680-12aa-40dd-8ec5-ebebf99916e5 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:41.639768 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2da680-12aa-40dd-8ec5-ebebf99916e5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:41.640073 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2da680-12aa-40dd-8ec5-ebebf99916e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:41.641538 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2da680-12aa-40dd-8ec5-ebebf99916e5 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:41.660145 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 371/715] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Sep 12 14:00:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:00:41.667998 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2da680-12aa-40dd-8ec5-ebebf99916e5 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:41.710395 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f550b4-34c5-4c1d-babe-d4be0d8958b4 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:41.726979 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:41.777145 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4efd09-1f43-466a-9075-90d9bfa49a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:41.777786 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4efd09-1f43-466a-9075-90d9bfa49a8a None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:41.778063 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4efd09-1f43-466a-9075-90d9bfa49a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:41.778330 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4efd09-1f43-466a-9075-90d9bfa49a8a None admin] PATH_INFO: `/v3/services/network` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:41.778941 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4efd09-1f43-466a-9075-90d9bfa49a8a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:41.780319 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4efd09-1f43-466a-9075-90d9bfa49a8a None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:41.793655 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-5b4efd09-1f43-466a-9075-90d9bfa49a8a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 14:00:41.793655 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:41.793655 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 14:00:41.793655 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 14:00:41.793655 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 14:00:41.793655 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 14:00:41.793655 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 14:00:41.793655 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:00:41.793655 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 14:00:41.793655 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:41.793655 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 14:00:41.793655 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:41.793655 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:41.793655 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:41.793655 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:41.793655 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:41.793655 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:41.794689 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:41.794689 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:41.794689 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:41.794689 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:41.794689 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:41.794689 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:41.794689 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 14:00:41.794689 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 14:00:41.794689 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 14:00:41.794689 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 14:00:41.794689 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:41.794689 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:41.794689 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:41.794689 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:41.794689 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:41.794689 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:41.795587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:41.795587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:41.795587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:41.795587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:41.795587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:41.795587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:41.795587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:41.795587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:41.795587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:41.795587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:41.795587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 14:00:41.795587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:00:41.795587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 14:00:41.795587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:41.795587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 345/716] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:00:41 2022] GET /identity/v3/services/network => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:41.802917 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4efd09-1f43-466a-9075-90d9bfa49a8a None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:41.814031 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:41.852771 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2da680-12aa-40dd-8ec5-ebebf99916e5 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:41.923661 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688701b-2a5f-4463-b168-29ec38ec9a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:41.924208 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688701b-2a5f-4463-b168-29ec38ec9a56 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:41.924544 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688701b-2a5f-4463-b168-29ec38ec9a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:41.924824 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688701b-2a5f-4463-b168-29ec38ec9a56 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:41.925470 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d688701b-2a5f-4463-b168-29ec38ec9a56 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:41.925772 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d688701b-2a5f-4463-b168-29ec38ec9a56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:41.927212 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d688701b-2a5f-4463-b168-29ec38ec9a56 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:41.933612 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 372/717] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Sep 12 14:00:41 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:41.946338 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4efd09-1f43-466a-9075-90d9bfa49a8a None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:42.003810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebf8b3c-b10e-4a16-8668-f95875dc5a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:42.004422 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf8b3c-b10e-4a16-8668-f95875dc5a4d None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:42.004694 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf8b3c-b10e-4a16-8668-f95875dc5a4d None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:42.004960 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf8b3c-b10e-4a16-8668-f95875dc5a4d None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:42.005590 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebf8b3c-b10e-4a16-8668-f95875dc5a4d None admin] RBAC: Adding query filter params (name=network) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:42.005884 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebf8b3c-b10e-4a16-8668-f95875dc5a4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:42.007276 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebf8b3c-b10e-4a16-8668-f95875dc5a4d None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:42.013010 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 346/718] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Sep 12 14:00:41 2022] GET /identity/v3/services?name=network => generated 124 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:42.020310 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688701b-2a5f-4463-b168-29ec38ec9a56 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:42.145030 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688701b-2a5f-4463-b168-29ec38ec9a56 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:42.181884 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba53b1-934f-4f35-a1ed-a4092b478a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:42.182421 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba53b1-934f-4f35-a1ed-a4092b478a62 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:42.182700 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba53b1-934f-4f35-a1ed-a4092b478a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:42.183028 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba53b1-934f-4f35-a1ed-a4092b478a62 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:42.183660 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ba53b1-934f-4f35-a1ed-a4092b478a62 None admin] RBAC: Adding query filter params (type=network) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:42.183960 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ba53b1-934f-4f35-a1ed-a4092b478a62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:42.185315 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ba53b1-934f-4f35-a1ed-a4092b478a62 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:42.190387 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 373/719] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Sep 12 14:00:42 2022] GET /identity/v3/services?type=network => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:43.542296 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730dd03-5957-429e-b4ca-015e66657adb None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:43.542876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730dd03-5957-429e-b4ca-015e66657adb None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:43.542876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730dd03-5957-429e-b4ca-015e66657adb None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:43.544420 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 347/720] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:43.549890 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000d2e6-e4b8-49cd-9de6-fc73e1674b99 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:43.550299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000d2e6-e4b8-49cd-9de6-fc73e1674b99 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:43.550463 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000d2e6-e4b8-49cd-9de6-fc73e1674b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:43.568412 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:43.579792 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-2000d2e6-e4b8-49cd-9de6-fc73e1674b99 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:43.580990 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-2000d2e6-e4b8-49cd-9de6-fc73e1674b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:43.667936 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 374/721] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 14:00:43.987761 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d81c0-1884-491b-acbb-7bb2bb359343 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:43.988097 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d81c0-1884-491b-acbb-7bb2bb359343 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:43.988316 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d81c0-1884-491b-acbb-7bb2bb359343 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:43.989967 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 348/722] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:43.991029 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2000d2e6-e4b8-49cd-9de6-fc73e1674b99 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:44.004442 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228352b2-d62f-4e1f-bf49-fbd167be1f8f None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:44.004759 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228352b2-d62f-4e1f-bf49-fbd167be1f8f None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:44.004993 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228352b2-d62f-4e1f-bf49-fbd167be1f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:44.032082 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-228352b2-d62f-4e1f-bf49-fbd167be1f8f None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:44.033493 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-228352b2-d62f-4e1f-bf49-fbd167be1f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:44.078543 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:44.103474 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2000d2e6-e4b8-49cd-9de6-fc73e1674b99 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:44.138779 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7134380-a123-4d0e-a029-38c59a17b8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:44.139768 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7134380-a123-4d0e-a029-38c59a17b8a4 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:44.139883 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7134380-a123-4d0e-a029-38c59a17b8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:44.140085 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7134380-a123-4d0e-a029-38c59a17b8a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:44.151046 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:44.159083 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7134380-a123-4d0e-a029-38c59a17b8a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:44.165024 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-a7134380-a123-4d0e-a029-38c59a17b8a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:00:44.165024 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:44.165024 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:44.165024 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:44.165024 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:44.165024 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:44.165024 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:44.165024 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:44.165024 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:44.165024 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:44.165024 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:44.165024 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:44.165024 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:44.165024 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:44.165024 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:44.165024 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:44.165963 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:44.165963 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:44.165963 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:44.165963 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:44.165963 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:44.165963 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:44.165963 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:44.165963 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:44.165963 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:44.165963 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:44.165963 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:44.165963 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:44.165963 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:44.165963 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:44.165963 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:44.165963 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:44.166777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:44.166777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:44.166777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:44.166777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:44.166777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:44.166777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:44.166777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:44.166777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:44.166777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:44.166777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:44.166777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:00:44.166777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:44.166777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 375/723] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 12 14:00:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:44.168167 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e330bc-65b2-437a-9958-e240876da7e1 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:44.168652 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e330bc-65b2-437a-9958-e240876da7e1 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:44.170894 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e330bc-65b2-437a-9958-e240876da7e1 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:44.171187 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 376/724] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:44.176813 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e330bc-65b2-437a-9958-e240876da7e1 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:44.183107 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 349/725] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:44.185122 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ec2660-9ef1-4153-af08-ac9768aa2a71 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:44.185785 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ec2660-9ef1-4153-af08-ac9768aa2a71 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:44.186445 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ec2660-9ef1-4153-af08-ac9768aa2a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:44.208034 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-35ec2660-9ef1-4153-af08-ac9768aa2a71 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:44.209543 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-35ec2660-9ef1-4153-af08-ac9768aa2a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:44.339112 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 350/726] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:44.364496 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e330bc-65b2-437a-9958-e240876da7e1 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:44.411712 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd9d6db-5262-480a-ba06-0dace40a6ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:44.412332 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd9d6db-5262-480a-ba06-0dace40a6ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:44.412613 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd9d6db-5262-480a-ba06-0dace40a6ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:44.412885 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd9d6db-5262-480a-ba06-0dace40a6ad3 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:44.413555 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd9d6db-5262-480a-ba06-0dace40a6ad3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:44.413860 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd9d6db-5262-480a-ba06-0dace40a6ad3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:44.415822 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd9d6db-5262-480a-ba06-0dace40a6ad3 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:44.432214 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 377/727] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 14:00:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:44.436028 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ec2660-9ef1-4153-af08-ac9768aa2a71 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:44.511478 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd9d6db-5262-480a-ba06-0dace40a6ad3 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:44.593679 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:44.605785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:44.609457 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ec2660-9ef1-4153-af08-ac9768aa2a71 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:44.654059 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd9d6db-5262-480a-ba06-0dace40a6ad3 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:44.658359 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4934e0d-7369-4f1b-815c-3c738fe3db9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:44.659276 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4934e0d-7369-4f1b-815c-3c738fe3db9c None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:44.659749 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4934e0d-7369-4f1b-815c-3c738fe3db9c None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:44.660220 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4934e0d-7369-4f1b-815c-3c738fe3db9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:44.674294 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4934e0d-7369-4f1b-815c-3c738fe3db9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:44.676916 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4934e0d-7369-4f1b-815c-3c738fe3db9c None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:44.692707 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a5f062-d79c-4d48-8f49-d31939c39a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:44.693667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a5f062-d79c-4d48-8f49-d31939c39a35 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:44.694140 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a5f062-d79c-4d48-8f49-d31939c39a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:44.694636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a5f062-d79c-4d48-8f49-d31939c39a35 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:44.710899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-d4934e0d-7369-4f1b-815c-3c738fe3db9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:44.710899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:44.710899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:44.710899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:44.710899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:44.710899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:44.710899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:44.710899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:44.710899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:44.710899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:44.710899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:44.710899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:44.710899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:44.710899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:44.710899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:44.710899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:44.712384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:44.712384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:44.712384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:44.712384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:44.712384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:44.712384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:44.712384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:44.712384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:44.712384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:44.712384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:44.712384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:44.712384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:44.712384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:44.712384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:44.712384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:44.712384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:44.712384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:44.713269 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:44.713269 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:44.713269 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:44.713269 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 351/728] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:00:44.713269 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5314352-b717-4929-b8fd-2068409a35ba None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:44.713269 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5314352-b717-4929-b8fd-2068409a35ba None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:44.713269 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5314352-b717-4929-b8fd-2068409a35ba None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:44.715381 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 352/729] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:44.716685 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01738a8d-0c2b-4aa6-9515-772b721c17f3 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:44.716973 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01738a8d-0c2b-4aa6-9515-772b721c17f3 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:44.717395 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01738a8d-0c2b-4aa6-9515-772b721c17f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:44.721402 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a5f062-d79c-4d48-8f49-d31939c39a35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:44.727299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-d5a5f062-d79c-4d48-8f49-d31939c39a35 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:00:44.727299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:44.727299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:44.727299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:44.727299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:44.727299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:44.727299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:44.727299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:44.727299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:44.727299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:44.727299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:44.727299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:44.727299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:44.727299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:44.727299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:44.727299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:44.728240 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:44.728240 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:44.728240 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:44.728240 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:44.728240 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:44.728240 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:44.728240 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:44.728240 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:44.728240 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:44.728240 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:44.728240 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:44.728240 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:44.728240 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:44.728240 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:44.728240 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:44.728240 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:44.729086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:44.729086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:44.729086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:44.729086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:44.729086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:44.729086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:44.729086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:44.729086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:44.729086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:44.729086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:44.729086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:00:44.729086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:44.731745 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 378/730] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 12 14:00:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:00:44.737547 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a5f062-d79c-4d48-8f49-d31939c39a35 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:44.737859 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-01738a8d-0c2b-4aa6-9515-772b721c17f3 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:44.739912 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-01738a8d-0c2b-4aa6-9515-772b721c17f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:44.823111 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 353/731] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:44.823969 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01738a8d-0c2b-4aa6-9515-772b721c17f3 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:44.869668 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a5f062-d79c-4d48-8f49-d31939c39a35 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:44.892054 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:44.908458 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:44.911616 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d3519-7530-44be-9e9f-c03114228b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:44.912341 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d3519-7530-44be-9e9f-c03114228b57 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:44.912699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d3519-7530-44be-9e9f-c03114228b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:44.913034 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d3519-7530-44be-9e9f-c03114228b57 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:44.913757 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d3519-7530-44be-9e9f-c03114228b57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:44.914148 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d3519-7530-44be-9e9f-c03114228b57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:44.915609 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d3519-7530-44be-9e9f-c03114228b57 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:44.933507 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 379/732] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:44.934674 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d3519-7530-44be-9e9f-c03114228b57 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:44.946824 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01738a8d-0c2b-4aa6-9515-772b721c17f3 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:44.987813 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1e79f-7c72-4e7a-b1da-6c264b78bbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:44.988508 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1e79f-7c72-4e7a-b1da-6c264b78bbce None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:44.988886 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1e79f-7c72-4e7a-b1da-6c264b78bbce None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:44.989238 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1e79f-7c72-4e7a-b1da-6c264b78bbce None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:44.990516 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e1e79f-7c72-4e7a-b1da-6c264b78bbce None admin] RBAC: Adding query filter params (name=service) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:44.990939 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e1e79f-7c72-4e7a-b1da-6c264b78bbce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:44.992863 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e1e79f-7c72-4e7a-b1da-6c264b78bbce None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:45.003617 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 354/733] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 12 14:00:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:45.008282 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1e79f-7c72-4e7a-b1da-6c264b78bbce None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:45.071894 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d3519-7530-44be-9e9f-c03114228b57 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:45.108305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07364bf-3c7c-4f7a-b8cc-6d65b9d5ba04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:45.109243 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07364bf-3c7c-4f7a-b8cc-6d65b9d5ba04 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:45.109725 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07364bf-3c7c-4f7a-b8cc-6d65b9d5ba04 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:45.110183 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07364bf-3c7c-4f7a-b8cc-6d65b9d5ba04 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:45.121496 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1e79f-7c72-4e7a-b1da-6c264b78bbce None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:45.121757 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07364bf-3c7c-4f7a-b8cc-6d65b9d5ba04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:45.126533 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-b07364bf-3c7c-4f7a-b8cc-6d65b9d5ba04 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:00:45.126533 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:45.126533 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:45.126533 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:45.126533 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:45.126533 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:45.126533 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:45.126533 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:45.126533 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:45.126533 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:45.126533 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:45.126533 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:45.126533 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:45.126533 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:45.126533 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:45.126533 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:45.127701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:45.127701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:45.127701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:45.127701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:45.127701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:45.127701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:45.127701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:45.127701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:45.127701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:45.127701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:45.127701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:45.127701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:45.127701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:45.127701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:45.127701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:45.127701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:45.128694 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:45.128694 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:45.128694 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:45.128694 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:45.128694 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:45.128694 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:45.128694 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:45.128694 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:45.128694 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:45.128694 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:45.128694 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:00:45.128694 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:45.128694 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 380/734] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 12 14:00:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:45.128694 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07364bf-3c7c-4f7a-b8cc-6d65b9d5ba04 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:45.157420 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db7c334-0684-4adb-854d-1ca2f49691d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:45.158038 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7c334-0684-4adb-854d-1ca2f49691d1 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:45.158309 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7c334-0684-4adb-854d-1ca2f49691d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:45.158616 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7c334-0684-4adb-854d-1ca2f49691d1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:45.160430 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:45.167330 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db7c334-0684-4adb-854d-1ca2f49691d1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:45.168393 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:45.168773 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db7c334-0684-4adb-854d-1ca2f49691d1 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:45.180465 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-3db7c334-0684-4adb-854d-1ca2f49691d1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 12 14:00:45.180465 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:45.180465 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:45.180465 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:45.180465 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:45.180465 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:45.180465 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:45.180465 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:45.180465 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:45.180465 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:45.180465 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:45.180465 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:45.180465 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:00:45.180465 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:00:45.180465 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:00:45.180465 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:00:45.181587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:45.181587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:45.181587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:00:45.181587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:45.181587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:00:45.181587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:45.181587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:45.181587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:45.181587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:45.181587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:45.181587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:45.181587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:45.181587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:45.181587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:45.181587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:45.181587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:45.181587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:45.181587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:45.187185 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:45.187185 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:45.187185 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:00:45.187185 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:00:45.187185 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:00:45.187185 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:00:45.187185 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:00:45.187185 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:00:45.187185 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 12 14:00:45.187185 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:45.187185 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 355/735] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 12 14:00:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:45.187185 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2781d-693b-45ba-81b6-c52bbc589c6a None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:45.187185 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2781d-693b-45ba-81b6-c52bbc589c6a None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:45.187964 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2781d-693b-45ba-81b6-c52bbc589c6a None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:45.187964 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 356/736] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:45.187964 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef2781d-693b-45ba-81b6-c52bbc589c6a None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:45.232840 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07364bf-3c7c-4f7a-b8cc-6d65b9d5ba04 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:45.266711 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d818d-b987-424b-8670-2c36a711faef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:45.267363 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d818d-b987-424b-8670-2c36a711faef None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:45.267645 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d818d-b987-424b-8670-2c36a711faef None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:45.267916 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d818d-b987-424b-8670-2c36a711faef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:45.277736 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64d818d-b987-424b-8670-2c36a711faef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:45.279186 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64d818d-b987-424b-8670-2c36a711faef None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:45.285617 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef2781d-693b-45ba-81b6-c52bbc589c6a None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:45.293519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-c64d818d-b987-424b-8670-2c36a711faef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:45.293519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:45.293519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:45.293519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:45.293519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:45.293519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:45.293519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:45.293519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:45.293519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:45.293519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:45.293519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:45.293519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:45.293519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:45.293519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:45.293519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:45.293519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:45.294500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:45.294500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:45.294500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:45.294500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:45.294500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:45.294500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:45.294500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:45.294500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:45.294500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:45.294500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:45.294500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:45.294500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:45.294500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:45.294500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:45.294500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:45.294500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:45.294500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:45.295454 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:45.295454 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:45.295454 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:45.295454 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 381/737] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:45.295454 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d818d-b987-424b-8670-2c36a711faef None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:45.323461 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5274156-4363-46a9-bb2c-257365e2473c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:45.324069 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5274156-4363-46a9-bb2c-257365e2473c None admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:45.324344 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5274156-4363-46a9-bb2c-257365e2473c None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:45.324613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5274156-4363-46a9-bb2c-257365e2473c None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:45.325260 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5274156-4363-46a9-bb2c-257365e2473c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:45.326629 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5274156-4363-46a9-bb2c-257365e2473c None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:45.342756 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 357/738] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Sep 12 14:00:45 2022] POST /identity/v3/services => generated 251 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:00:45.343591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5274156-4363-46a9-bb2c-257365e2473c None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:45.402313 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:45.413116 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d818d-b987-424b-8670-2c36a711faef None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:45.415619 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:45.469048 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e16d7-e2a9-460a-a6e6-6c871980f6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:45.469714 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e16d7-e2a9-460a-a6e6-6c871980f6e8 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:45.469997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e16d7-e2a9-460a-a6e6-6c871980f6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:45.470265 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e16d7-e2a9-460a-a6e6-6c871980f6e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:45.470994 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e16d7-e2a9-460a-a6e6-6c871980f6e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:45.471296 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e16d7-e2a9-460a-a6e6-6c871980f6e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:45.472681 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e16d7-e2a9-460a-a6e6-6c871980f6e8 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:45.474749 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5274156-4363-46a9-bb2c-257365e2473c None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:45.487493 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 382/739] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 14:00:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:45.487493 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b425f0-6651-4669-88e7-a6af52c07e4f None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:45.487493 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b425f0-6651-4669-88e7-a6af52c07e4f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:45.487493 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b425f0-6651-4669-88e7-a6af52c07e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:45.506283 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-e5b425f0-6651-4669-88e7-a6af52c07e4f None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:45.507607 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e5b425f0-6651-4669-88e7-a6af52c07e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:45.521965 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fe9acf-5e4a-4bd4-93e6-b4592d8505ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:45.522572 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe9acf-5e4a-4bd4-93e6-b4592d8505ba None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:45.522869 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe9acf-5e4a-4bd4-93e6-b4592d8505ba None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:45.523161 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe9acf-5e4a-4bd4-93e6-b4592d8505ba None admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:45.523910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fe9acf-5e4a-4bd4-93e6-b4592d8505ba None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:45.524214 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fe9acf-5e4a-4bd4-93e6-b4592d8505ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:45.525671 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fe9acf-5e4a-4bd4-93e6-b4592d8505ba None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:45.545202 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 358/740] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Sep 12 14:00:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:45.546052 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fe9acf-5e4a-4bd4-93e6-b4592d8505ba None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:45.590965 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 383/741] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:45 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 14:00:45.591816 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b425f0-6651-4669-88e7-a6af52c07e4f None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:45.676032 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fe9acf-5e4a-4bd4-93e6-b4592d8505ba None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:45.688784 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:45.691728 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:45.718344 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b425f0-6651-4669-88e7-a6af52c07e4f None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:45.729718 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea8ab5-4926-4194-b4ab-6b7aabe7563e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:45.730322 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea8ab5-4926-4194-b4ab-6b7aabe7563e None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:45.730593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea8ab5-4926-4194-b4ab-6b7aabe7563e None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:45.730881 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea8ab5-4926-4194-b4ab-6b7aabe7563e None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:45.731579 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea8ab5-4926-4194-b4ab-6b7aabe7563e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:45.731880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea8ab5-4926-4194-b4ab-6b7aabe7563e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:45.733233 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea8ab5-4926-4194-b4ab-6b7aabe7563e None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:45.746943 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 359/742] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:45.747782 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea8ab5-4926-4194-b4ab-6b7aabe7563e None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:45.761368 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85fccb4-aa93-4277-a1bf-2f043edd24eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:45.762302 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85fccb4-aa93-4277-a1bf-2f043edd24eb None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:45.763071 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85fccb4-aa93-4277-a1bf-2f043edd24eb None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:45.764009 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85fccb4-aa93-4277-a1bf-2f043edd24eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:45.776543 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85fccb4-aa93-4277-a1bf-2f043edd24eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:45.778037 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85fccb4-aa93-4277-a1bf-2f043edd24eb None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:45.795118 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-b85fccb4-aa93-4277-a1bf-2f043edd24eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:45.795118 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:45.795118 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:45.795118 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:45.795118 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:45.795118 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:45.795118 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:45.795118 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:45.795118 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:45.795118 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:45.795118 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:45.795118 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:45.795118 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:45.795118 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:45.795118 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:45.795118 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:45.796085 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:45.796085 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:45.796085 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:45.796085 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:45.796085 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:45.796085 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:45.796085 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:45.796085 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:45.796085 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:45.796085 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:45.796085 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:45.796085 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:45.796085 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:45.796085 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:45.796085 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:45.796085 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:45.796085 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:45.796967 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:45.796967 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:45.796967 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:45.796967 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 384/743] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:45.797606 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85fccb4-aa93-4277-a1bf-2f043edd24eb None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:45.881442 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea8ab5-4926-4194-b4ab-6b7aabe7563e None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:45.907634 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85fccb4-aa93-4277-a1bf-2f043edd24eb None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:45.921145 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c76fd7-ee4a-4d4a-879b-7f80e36a791f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:45.921752 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c76fd7-ee4a-4d4a-879b-7f80e36a791f None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:45.922029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c76fd7-ee4a-4d4a-879b-7f80e36a791f None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:45.922329 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c76fd7-ee4a-4d4a-879b-7f80e36a791f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:45.936846 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:45.943960 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abe731d-3d0f-4500-8050-161873e3d4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:45.944608 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe731d-3d0f-4500-8050-161873e3d4a0 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:45.944884 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe731d-3d0f-4500-8050-161873e3d4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:45.945734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe731d-3d0f-4500-8050-161873e3d4a0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:45.953540 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abe731d-3d0f-4500-8050-161873e3d4a0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:45.954984 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abe731d-3d0f-4500-8050-161873e3d4a0 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:45.959137 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:45.966612 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-2abe731d-3d0f-4500-8050-161873e3d4a0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 14:00:45.966612 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:45.966612 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:45.966612 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:45.966612 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:45.966612 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:45.966612 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:45.966612 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:45.966612 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:45.966612 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:45.966612 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:45.966612 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:45.966612 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:00:45.966612 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:00:45.966612 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:00:45.966612 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:00:45.967606 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:45.967606 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:45.967606 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:00:45.967606 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:45.967606 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:00:45.967606 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:45.967606 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:45.967606 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:45.967606 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:45.967606 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:45.967606 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:45.967606 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:45.967606 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:45.967606 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:45.967606 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:45.967606 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:45.967606 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:45.967606 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:45.973265 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:45.973265 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:45.973265 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:00:45.973265 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:00:45.973265 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:00:45.973265 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:00:45.973265 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:00:45.973265 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:00:45.973265 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 14:00:45.973265 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:45.973265 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 385/744] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Sep 12 14:00:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:45.973265 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abe731d-3d0f-4500-8050-161873e3d4a0 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:46.002270 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c76fd7-ee4a-4d4a-879b-7f80e36a791f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:46.004241 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c76fd7-ee4a-4d4a-879b-7f80e36a791f None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:46.074315 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abe731d-3d0f-4500-8050-161873e3d4a0 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:46.109008 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 360/745] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:00:45 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 14:00:46.110387 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c76fd7-ee4a-4d4a-879b-7f80e36a791f None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:46.112649 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868dcc2-bf6c-403b-8723-2dafea67e6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:46.113728 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868dcc2-bf6c-403b-8723-2dafea67e6b0 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:46.113872 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868dcc2-bf6c-403b-8723-2dafea67e6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:46.114124 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868dcc2-bf6c-403b-8723-2dafea67e6b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:46.115666 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2868dcc2-bf6c-403b-8723-2dafea67e6b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:46.115984 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2868dcc2-bf6c-403b-8723-2dafea67e6b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:46.117919 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2868dcc2-bf6c-403b-8723-2dafea67e6b0 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:46.131727 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 386/746] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:46.132548 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868dcc2-bf6c-403b-8723-2dafea67e6b0 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:46.184930 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:46.222808 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c76fd7-ee4a-4d4a-879b-7f80e36a791f None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:46.262189 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b8392-f02a-484f-9e63-5ac0dc5315ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:46.262825 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b8392-f02a-484f-9e63-5ac0dc5315ae None admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:46.263152 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b8392-f02a-484f-9e63-5ac0dc5315ae None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:46.263395 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b8392-f02a-484f-9e63-5ac0dc5315ae None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:46.264088 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b8392-f02a-484f-9e63-5ac0dc5315ae None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:46.265536 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b8392-f02a-484f-9e63-5ac0dc5315ae None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:46.286895 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 361/747] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Sep 12 14:00:46 2022] POST /identity/v3/services => generated 240 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:00:46.286895 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b8392-f02a-484f-9e63-5ac0dc5315ae None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:46.302594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:46.360682 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868dcc2-bf6c-403b-8723-2dafea67e6b0 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:46.411518 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757abae-b317-4944-9620-1b181d700cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:46.412228 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757abae-b317-4944-9620-1b181d700cf0 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:46.412501 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757abae-b317-4944-9620-1b181d700cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:46.412777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757abae-b317-4944-9620-1b181d700cf0 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:46.413520 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757abae-b317-4944-9620-1b181d700cf0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:46.413824 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757abae-b317-4944-9620-1b181d700cf0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:46.416015 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757abae-b317-4944-9620-1b181d700cf0 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:46.428172 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b8392-f02a-484f-9e63-5ac0dc5315ae None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:46.436480 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 387/748] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Sep 12 14:00:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:46.442093 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757abae-b317-4944-9620-1b181d700cf0 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:46.464104 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:46.477349 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d572716a-8068-4d5c-ad08-a064d0872b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:46.478428 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572716a-8068-4d5c-ad08-a064d0872b94 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:46.479281 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572716a-8068-4d5c-ad08-a064d0872b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:46.479714 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572716a-8068-4d5c-ad08-a064d0872b94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:46.499750 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d572716a-8068-4d5c-ad08-a064d0872b94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:46.501925 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d572716a-8068-4d5c-ad08-a064d0872b94 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:46.522327 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-d572716a-8068-4d5c-ad08-a064d0872b94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:46.522327 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:46.522327 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:46.522327 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:46.522327 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:46.522327 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:46.522327 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:46.522327 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:46.522327 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:46.522327 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:46.522327 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:46.522327 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:46.522327 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:46.522327 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:46.522327 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:46.522327 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:46.523970 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:46.523970 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:46.523970 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:46.523970 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:46.523970 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:46.523970 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:46.523970 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:46.523970 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:46.523970 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:46.523970 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:46.523970 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:46.523970 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:46.523970 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:46.523970 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:46.523970 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:46.523970 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:46.523970 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:46.525229 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:46.525229 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:46.525229 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:46.525229 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 362/749] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:46.525229 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d572716a-8068-4d5c-ad08-a064d0872b94 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:46.567217 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757abae-b317-4944-9620-1b181d700cf0 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:46.571125 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:46.607252 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6022f79-9905-48ec-a97f-4534326d14a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:46.607894 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6022f79-9905-48ec-a97f-4534326d14a1 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:46.608175 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6022f79-9905-48ec-a97f-4534326d14a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:46.608448 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6022f79-9905-48ec-a97f-4534326d14a1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:46.620287 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6022f79-9905-48ec-a97f-4534326d14a1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:46.621707 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6022f79-9905-48ec-a97f-4534326d14a1 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:46.635568 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-f6022f79-9905-48ec-a97f-4534326d14a1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 14:00:46.635568 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:46.635568 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:46.635568 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:46.635568 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:46.635568 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:46.635568 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:46.635568 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:46.635568 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:46.635568 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:46.635568 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:46.635568 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:46.635568 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:00:46.635568 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:00:46.635568 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:00:46.635568 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:00:46.636564 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:46.636564 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:46.636564 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:00:46.636564 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:46.636564 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:00:46.636564 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:46.636564 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:46.636564 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:46.636564 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:46.636564 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:46.636564 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:46.636564 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:46.636564 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:46.636564 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:46.636564 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:46.636564 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:46.636564 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:46.636564 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:46.638512 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:46.638512 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:46.638512 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:00:46.638512 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:00:46.638512 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:00:46.638512 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:00:46.638512 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:00:46.638512 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:00:46.638512 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 14:00:46.638512 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:46.638512 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 388/750] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Sep 12 14:00:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:46.638512 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6022f79-9905-48ec-a97f-4534326d14a1 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:46.648152 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d572716a-8068-4d5c-ad08-a064d0872b94 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:46.684513 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f97f28-7682-4961-bdf4-96500a65aca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:46.685146 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f97f28-7682-4961-bdf4-96500a65aca9 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:46.685407 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f97f28-7682-4961-bdf4-96500a65aca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:46.685658 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f97f28-7682-4961-bdf4-96500a65aca9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:46.743909 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:46.754146 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6022f79-9905-48ec-a97f-4534326d14a1 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:46.765919 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f97f28-7682-4961-bdf4-96500a65aca9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:46.767616 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f97f28-7682-4961-bdf4-96500a65aca9 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:46.804042 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcde8a5-b99a-453b-84a8-679044c3a11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:46.804952 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcde8a5-b99a-453b-84a8-679044c3a11c None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:46.805242 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcde8a5-b99a-453b-84a8-679044c3a11c None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:46.805513 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcde8a5-b99a-453b-84a8-679044c3a11c None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:46.807050 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcde8a5-b99a-453b-84a8-679044c3a11c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:46.807355 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcde8a5-b99a-453b-84a8-679044c3a11c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:46.808708 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcde8a5-b99a-453b-84a8-679044c3a11c None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:46.824149 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 389/751] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:46.824982 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcde8a5-b99a-453b-84a8-679044c3a11c None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:46.857706 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:46.881809 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 363/752] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:00:46 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 14:00:46.883094 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f97f28-7682-4961-bdf4-96500a65aca9 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:46.958152 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcde8a5-b99a-453b-84a8-679044c3a11c None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:46.970703 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:46.992163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f97f28-7682-4961-bdf4-96500a65aca9 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:47.004887 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce8289-17a3-4d28-80d7-608ec712b2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:47.005590 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce8289-17a3-4d28-80d7-608ec712b2c3 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:47.005976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce8289-17a3-4d28-80d7-608ec712b2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:47.006247 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce8289-17a3-4d28-80d7-608ec712b2c3 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:47.007002 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ce8289-17a3-4d28-80d7-608ec712b2c3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:47.007306 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ce8289-17a3-4d28-80d7-608ec712b2c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:47.009516 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ce8289-17a3-4d28-80d7-608ec712b2c3 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:47.031041 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10193b8-a497-435a-9b3a-6cf210cb094a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:47.032023 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10193b8-a497-435a-9b3a-6cf210cb094a None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:47.032510 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10193b8-a497-435a-9b3a-6cf210cb094a None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:47.033009 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10193b8-a497-435a-9b3a-6cf210cb094a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:47.038083 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 390/753] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Sep 12 14:00:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:47.038083 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce8289-17a3-4d28-80d7-608ec712b2c3 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:47.045596 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10193b8-a497-435a-9b3a-6cf210cb094a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:47.047410 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10193b8-a497-435a-9b3a-6cf210cb094a None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:47.059949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-a10193b8-a497-435a-9b3a-6cf210cb094a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:47.059949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:47.059949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:47.059949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:47.059949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:47.059949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:47.059949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:47.059949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:47.059949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:47.059949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:47.059949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:47.059949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:47.059949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:00:47.059949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:00:47.059949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:00:47.060852 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:00:47.060852 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:47.060852 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:47.060852 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:47.060852 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:47.060852 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:47.060852 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:47.060852 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:47.060852 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:47.060852 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:47.060852 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:47.060852 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:47.060852 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:47.060852 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:47.060852 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:47.060852 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:47.060852 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:47.061738 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:00:47.061738 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:00:47.061738 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:00:47.061738 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:00:47.061738 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:00:47.061738 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:00:47.061738 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:47.061738 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:47.065079 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 364/754] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:00:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:47.066213 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10193b8-a497-435a-9b3a-6cf210cb094a None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:47.139636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:47.168835 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce8289-17a3-4d28-80d7-608ec712b2c3 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:47.176815 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10193b8-a497-435a-9b3a-6cf210cb094a None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:47.213665 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd77d9a-cb78-470b-b774-fdd02a88be0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:47.214058 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391c2d7-2f54-4072-9589-8e081e73f95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:47.214696 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391c2d7-2f54-4072-9589-8e081e73f95f None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:47.215011 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391c2d7-2f54-4072-9589-8e081e73f95f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:47.215277 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391c2d7-2f54-4072-9589-8e081e73f95f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:47.215597 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd77d9a-cb78-470b-b774-fdd02a88be0e None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:47.216231 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd77d9a-cb78-470b-b774-fdd02a88be0e None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:47.216680 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd77d9a-cb78-470b-b774-fdd02a88be0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:47.225944 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:47.229058 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9391c2d7-2f54-4072-9589-8e081e73f95f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:47.229774 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd77d9a-cb78-470b-b774-fdd02a88be0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:47.230708 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9391c2d7-2f54-4072-9589-8e081e73f95f None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:47.232030 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd77d9a-cb78-470b-b774-fdd02a88be0e None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:47.248818 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-8bd77d9a-cb78-470b-b774-fdd02a88be0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:47.248818 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:47.248818 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:47.248818 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:47.248818 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:47.248818 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:47.248818 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:47.248818 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:47.248818 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:47.248818 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:47.248818 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:47.248818 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:47.248818 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:47.248818 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:47.248818 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:47.248818 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:47.249884 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:47.249884 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:47.249884 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:47.249884 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:47.249884 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:47.249884 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:47.249884 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:47.249884 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:47.249884 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:47.249884 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:47.249884 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:47.249884 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:47.249884 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:47.249884 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:47.249884 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:47.249884 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:47.249884 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:47.250796 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:47.250796 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:47.250796 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:47.251005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-9391c2d7-2f54-4072-9589-8e081e73f95f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:47.251005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:47.251005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:47.251005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:47.251005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:47.251005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:47.251005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:47.251005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:47.251005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:47.251005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:47.251005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:47.251005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:47.251005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:47.251005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:47.251005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:47.251005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:47.251834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:47.251834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:47.251834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:47.251834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:47.251834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:47.251834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:47.251834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:47.251834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:47.251834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:47.251834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:47.251834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:47.251834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:47.251834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:47.251834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:47.251834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:47.251834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:47.251834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:47.253507 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:47.253507 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:47.253507 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:47.253507 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 391/755] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:47.253507 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391c2d7-2f54-4072-9589-8e081e73f95f None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:47.259936 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 365/756] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:47.261013 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd77d9a-cb78-470b-b774-fdd02a88be0e None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:47.372488 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd77d9a-cb78-470b-b774-fdd02a88be0e None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:47.383165 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391c2d7-2f54-4072-9589-8e081e73f95f None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:47.410061 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5a9c3a-aebf-4a8e-a823-efa2e1f2e49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:47.411047 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a9c3a-aebf-4a8e-a823-efa2e1f2e49d None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:47.411618 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a9c3a-aebf-4a8e-a823-efa2e1f2e49d None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:47.412110 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a9c3a-aebf-4a8e-a823-efa2e1f2e49d None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:47.412998 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5a9c3a-aebf-4a8e-a823-efa2e1f2e49d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:47.413498 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5a9c3a-aebf-4a8e-a823-efa2e1f2e49d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:47.416017 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5a9c3a-aebf-4a8e-a823-efa2e1f2e49d None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:47.424666 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0948cf33-a18d-4456-b1b8-58eea287cff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:47.425275 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0948cf33-a18d-4456-b1b8-58eea287cff3 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:47.425549 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0948cf33-a18d-4456-b1b8-58eea287cff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:47.425815 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0948cf33-a18d-4456-b1b8-58eea287cff3 None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:47.426498 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0948cf33-a18d-4456-b1b8-58eea287cff3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:47.426797 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0948cf33-a18d-4456-b1b8-58eea287cff3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:47.429012 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0948cf33-a18d-4456-b1b8-58eea287cff3 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:47.430599 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:47.432426 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 366/757] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:47.433577 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5a9c3a-aebf-4a8e-a823-efa2e1f2e49d None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:47.444451 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 392/758] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 12 14:00:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:47.445527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0948cf33-a18d-4456-b1b8-58eea287cff3 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:47.498163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:47.557863 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5a9c3a-aebf-4a8e-a823-efa2e1f2e49d None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:47.571258 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0948cf33-a18d-4456-b1b8-58eea287cff3 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:47.593926 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c40339-9b9f-41f9-b13e-9b489492ba01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:47.594525 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c40339-9b9f-41f9-b13e-9b489492ba01 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:47.594798 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c40339-9b9f-41f9-b13e-9b489492ba01 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:47.595092 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c40339-9b9f-41f9-b13e-9b489492ba01 None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:47.595718 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c40339-9b9f-41f9-b13e-9b489492ba01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:47.596015 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c40339-9b9f-41f9-b13e-9b489492ba01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:47.597368 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c40339-9b9f-41f9-b13e-9b489492ba01 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:47.608860 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 367/759] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:47.609641 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c40339-9b9f-41f9-b13e-9b489492ba01 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:47.611991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6228a15e-28da-4ca2-b23d-101a64aa8e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:47.612620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6228a15e-28da-4ca2-b23d-101a64aa8e8d None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:47.612899 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6228a15e-28da-4ca2-b23d-101a64aa8e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:47.613268 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6228a15e-28da-4ca2-b23d-101a64aa8e8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:47.620566 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6228a15e-28da-4ca2-b23d-101a64aa8e8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:47.622122 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6228a15e-28da-4ca2-b23d-101a64aa8e8d None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:47.634288 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-6228a15e-28da-4ca2-b23d-101a64aa8e8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:47.634288 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:47.634288 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:47.634288 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:47.634288 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:47.634288 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:47.634288 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:47.634288 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:47.634288 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:47.634288 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:47.634288 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:47.634288 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:47.634288 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:00:47.634288 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:00:47.634288 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:00:47.635211 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:00:47.635211 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:47.635211 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:47.635211 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:47.635211 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:47.635211 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:47.635211 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:47.635211 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:47.635211 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:47.635211 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:47.635211 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:47.635211 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:47.635211 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:47.635211 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:47.635211 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:47.635211 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:47.635211 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:47.636117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:00:47.636117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:00:47.636117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:00:47.636117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:00:47.636117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:00:47.636117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:00:47.636117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:47.636117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:47.636117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 393/760] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:00:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:47.637028 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2e82c-63c6-44fc-bfab-661a57127517 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:47.637028 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2e82c-63c6-44fc-bfab-661a57127517 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:47.637320 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2e82c-63c6-44fc-bfab-661a57127517 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:47.642544 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 394/761] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:47.642544 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2e82c-63c6-44fc-bfab-661a57127517 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:47.701897 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:47.725571 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c40339-9b9f-41f9-b13e-9b489492ba01 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:47.738959 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:47.748075 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2e82c-63c6-44fc-bfab-661a57127517 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:47.761491 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2b9455-3c67-4e8a-aa51-a6d2b7e58e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:47.762091 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b9455-3c67-4e8a-aa51-a6d2b7e58e10 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:47.762373 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b9455-3c67-4e8a-aa51-a6d2b7e58e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:47.762646 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b9455-3c67-4e8a-aa51-a6d2b7e58e10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:47.763398 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2b9455-3c67-4e8a-aa51-a6d2b7e58e10 None admin] RBAC: Adding query filter params (role.id=e14564f00daf48a28d4e32da8cd2efad, scope.project.id=f666a617f8d14eb18bbeb076032b82f2, user.id=25a87ea9c3c5409b97a35c4857ae3ca6) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:47.763714 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2b9455-3c67-4e8a-aa51-a6d2b7e58e10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:47.765143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2b9455-3c67-4e8a-aa51-a6d2b7e58e10 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:47.770408 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 368/762] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 14:00:47 2022] GET /identity/v3/role_assignments?user.id=25a87ea9c3c5409b97a35c4857ae3ca6&scope.project.id=f666a617f8d14eb18bbeb076032b82f2&role.id=e14564f00daf48a28d4e32da8cd2efad => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:47.771220 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2b9455-3c67-4e8a-aa51-a6d2b7e58e10 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:47.796760 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703cf693-a65c-4f94-b8f8-403de9ec14ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:47.798114 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703cf693-a65c-4f94-b8f8-403de9ec14ef None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:47.798645 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703cf693-a65c-4f94-b8f8-403de9ec14ef None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:47.799094 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703cf693-a65c-4f94-b8f8-403de9ec14ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:47.809111 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703cf693-a65c-4f94-b8f8-403de9ec14ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:47.810576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703cf693-a65c-4f94-b8f8-403de9ec14ef None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:47.828777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-703cf693-a65c-4f94-b8f8-403de9ec14ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:47.828777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:47.828777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:47.828777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:47.828777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:47.828777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:47.828777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:47.828777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:47.828777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:47.828777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:47.828777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:47.828777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:47.828777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:00:47.828777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:00:47.828777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:00:47.829938 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:00:47.829938 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:47.829938 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:47.829938 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:47.829938 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:47.829938 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:47.829938 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:47.829938 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:47.829938 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:47.829938 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:47.829938 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:47.829938 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:47.829938 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:47.829938 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:47.829938 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:47.829938 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:47.829938 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:47.831090 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:00:47.831090 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:00:47.831090 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:00:47.831090 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:00:47.831090 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:00:47.831090 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:00:47.831090 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:47.831090 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:47.836129 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 395/763] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:00:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:47.838574 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b76be0-347c-485f-9589-744088781707 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:47.839027 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b76be0-347c-485f-9589-744088781707 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:47.839440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b76be0-347c-485f-9589-744088781707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:47.859835 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-78b76be0-347c-485f-9589-744088781707 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:47.861319 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-78b76be0-347c-485f-9589-744088781707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:47.901499 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2b9455-3c67-4e8a-aa51-a6d2b7e58e10 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:47.943740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6be1dc-2c1a-4f07-8302-559520dcd054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:47.944379 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6be1dc-2c1a-4f07-8302-559520dcd054 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:47.944654 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6be1dc-2c1a-4f07-8302-559520dcd054 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:47.944924 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6be1dc-2c1a-4f07-8302-559520dcd054 None admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:47.946187 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6be1dc-2c1a-4f07-8302-559520dcd054 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:47.946745 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6be1dc-2c1a-4f07-8302-559520dcd054 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:47.948378 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6be1dc-2c1a-4f07-8302-559520dcd054 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:47.955885 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 396/764] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:00:47.957180 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b76be0-347c-485f-9589-744088781707 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:47.977885 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 369/765] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 12 14:00:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:47.983845 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6be1dc-2c1a-4f07-8302-559520dcd054 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:48.025774 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:48.042574 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:48.131847 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b76be0-347c-485f-9589-744088781707 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:48.149985 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6be1dc-2c1a-4f07-8302-559520dcd054 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:48.178393 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92297f-5f7e-4bac-9088-c1e565a911f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:48.179397 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92297f-5f7e-4bac-9088-c1e565a911f9 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:48.180455 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92297f-5f7e-4bac-9088-c1e565a911f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:48.180946 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92297f-5f7e-4bac-9088-c1e565a911f9 None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:48.181943 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad92297f-5f7e-4bac-9088-c1e565a911f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:48.182449 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad92297f-5f7e-4bac-9088-c1e565a911f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:48.184089 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad92297f-5f7e-4bac-9088-c1e565a911f9 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:48.197541 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 397/766] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 12 14:00:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:48.202887 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92297f-5f7e-4bac-9088-c1e565a911f9 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:48.234637 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dccbdf-3018-4f33-9a2d-858ee690c86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:48.236663 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dccbdf-3018-4f33-9a2d-858ee690c86b None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:48.237165 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dccbdf-3018-4f33-9a2d-858ee690c86b None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:48.238567 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dccbdf-3018-4f33-9a2d-858ee690c86b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:48.239529 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dccbdf-3018-4f33-9a2d-858ee690c86b None admin] RBAC: Adding query filter params (role.id=1c413f4e78ef45b7a821bb884f9cea07, scope.project.id=f666a617f8d14eb18bbeb076032b82f2, user.id=1405e758a3af4e8ba659f498cd28ace1) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:48.243676 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dccbdf-3018-4f33-9a2d-858ee690c86b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:48.245313 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dccbdf-3018-4f33-9a2d-858ee690c86b None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:48.254463 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 370/767] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 14:00:47 2022] GET /identity/v3/role_assignments?user.id=1405e758a3af4e8ba659f498cd28ace1&scope.project.id=f666a617f8d14eb18bbeb076032b82f2&role.id=1c413f4e78ef45b7a821bb884f9cea07 => generated 259 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:00:48.260166 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dccbdf-3018-4f33-9a2d-858ee690c86b None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:48.416875 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:48.421368 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dccbdf-3018-4f33-9a2d-858ee690c86b None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:48.437701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92297f-5f7e-4bac-9088-c1e565a911f9 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:48.444701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:48.465725 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd173e3-470c-4d7f-8333-1b826ae863c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:48.466855 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd173e3-470c-4d7f-8333-1b826ae863c6 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:48.467310 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd173e3-470c-4d7f-8333-1b826ae863c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:48.467780 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd173e3-470c-4d7f-8333-1b826ae863c6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:48.468541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd173e3-470c-4d7f-8333-1b826ae863c6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:48.470089 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd173e3-470c-4d7f-8333-1b826ae863c6 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:48.482627 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-dbd173e3-470c-4d7f-8333-1b826ae863c6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 14:00:48.482627 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:48.482627 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 14:00:48.482627 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 14:00:48.482627 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 14:00:48.482627 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 14:00:48.482627 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 14:00:48.482627 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:00:48.482627 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 14:00:48.482627 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:48.482627 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 14:00:48.482627 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:48.482627 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:48.482627 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:48.482627 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:48.482627 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:48.482627 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:48.483782 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:48.483782 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:48.483782 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:48.483782 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:48.483782 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:48.483782 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:48.483782 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 14:00:48.483782 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 14:00:48.483782 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 14:00:48.483782 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 14:00:48.483782 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:48.483782 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:48.483782 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:48.483782 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:48.483782 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:48.483782 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:48.484613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:48.484613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:48.484613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:48.484613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:48.484613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:48.484613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:48.484613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:48.484613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:48.484613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:48.484613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:48.484613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 14:00:48.484613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:00:48.484613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 14:00:48.484613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:48.487938 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 371/768] 10.222.0.42 () {56 vars in 1129 bytes} [Mon Sep 12 14:00:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:48.496748 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd173e3-470c-4d7f-8333-1b826ae863c6 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:48.517677 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07938af0-2ae1-4979-8651-a8e3d3866bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:48.518537 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07938af0-2ae1-4979-8651-a8e3d3866bbd None admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:48.518827 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07938af0-2ae1-4979-8651-a8e3d3866bbd None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:48.519131 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07938af0-2ae1-4979-8651-a8e3d3866bbd None admin] PATH_INFO: `/v3/projects/f666a617f8d14eb18bbeb076032b82f2/users/854ab2dcec5a48b18f409b33cb61b5da/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:48.521258 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07938af0-2ae1-4979-8651-a8e3d3866bbd None admin] RBAC: Authorizing `identity:create_grant(project_id=f666a617f8d14eb18bbeb076032b82f2, user_id=854ab2dcec5a48b18f409b33cb61b5da, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:48.522819 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07938af0-2ae1-4979-8651-a8e3d3866bbd None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:48.551607 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 398/769] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 12 14:00:48 2022] PUT /identity/v3/projects/f666a617f8d14eb18bbeb076032b82f2/users/854ab2dcec5a48b18f409b33cb61b5da/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:00:48.614146 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd173e3-470c-4d7f-8333-1b826ae863c6 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:48.655735 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc9371-b809-4bbb-8d53-0873cd5a1f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:48.656416 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc9371-b809-4bbb-8d53-0873cd5a1f04 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:48.656744 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc9371-b809-4bbb-8d53-0873cd5a1f04 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:48.657019 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc9371-b809-4bbb-8d53-0873cd5a1f04 None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:48.657672 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cc9371-b809-4bbb-8d53-0873cd5a1f04 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:48.658026 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cc9371-b809-4bbb-8d53-0873cd5a1f04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:48.659579 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cc9371-b809-4bbb-8d53-0873cd5a1f04 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:48.670133 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 372/770] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Sep 12 14:00:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:48.672074 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07938af0-2ae1-4979-8651-a8e3d3866bbd None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:48.786009 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07938af0-2ae1-4979-8651-a8e3d3866bbd None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:48.836272 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d14c4-a29c-4064-bdce-d7219b0a64ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:48.836832 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d14c4-a29c-4064-bdce-d7219b0a64ea None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:48.842865 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d14c4-a29c-4064-bdce-d7219b0a64ea None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:48.842865 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d14c4-a29c-4064-bdce-d7219b0a64ea None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:48.842865 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38d14c4-a29c-4064-bdce-d7219b0a64ea None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:48.842865 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38d14c4-a29c-4064-bdce-d7219b0a64ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:48.842865 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38d14c4-a29c-4064-bdce-d7219b0a64ea None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:48.851031 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 399/771] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Sep 12 14:00:48 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:48.853069 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc9371-b809-4bbb-8d53-0873cd5a1f04 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:48.856088 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0c140-61d1-4e12-b953-8c4067a8f2ca None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:48.856381 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0c140-61d1-4e12-b953-8c4067a8f2ca None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:48.856657 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0c140-61d1-4e12-b953-8c4067a8f2ca None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:48.858156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 400/772] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:48.863723 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8750d5b-6de7-4fa5-86c6-d018e9a9d64a None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:48.864009 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8750d5b-6de7-4fa5-86c6-d018e9a9d64a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:48.864287 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8750d5b-6de7-4fa5-86c6-d018e9a9d64a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:48.893232 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-e8750d5b-6de7-4fa5-86c6-d018e9a9d64a None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:48.894496 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e8750d5b-6de7-4fa5-86c6-d018e9a9d64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:48.922533 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:48.924635 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:48.980943 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 401/773] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:00:49.008395 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc9371-b809-4bbb-8d53-0873cd5a1f04 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:49.043507 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc23cf3-3ffe-4d30-968f-a7610eb49293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:49.044118 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc23cf3-3ffe-4d30-968f-a7610eb49293 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:49.044397 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc23cf3-3ffe-4d30-968f-a7610eb49293 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:49.044670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc23cf3-3ffe-4d30-968f-a7610eb49293 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:49.045370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc23cf3-3ffe-4d30-968f-a7610eb49293 None admin] RBAC: Adding query filter params (interface=public, service_id=297c797c40bd41b49ad8130c47f5b208, region_id=RegionOne) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:49.045674 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc23cf3-3ffe-4d30-968f-a7610eb49293 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:49.047053 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc23cf3-3ffe-4d30-968f-a7610eb49293 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:49.055193 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 373/774] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Sep 12 14:00:48 2022] GET /identity/v3/endpoints?service_id=297c797c40bd41b49ad8130c47f5b208&interface=public®ion_id=RegionOne => generated 194 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:00:49.063989 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8750d5b-6de7-4fa5-86c6-d018e9a9d64a None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:49.166699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8750d5b-6de7-4fa5-86c6-d018e9a9d64a None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:49.201748 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e200e94-e1b4-4a8e-8f2a-306bfce6aef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:49.202279 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e200e94-e1b4-4a8e-8f2a-306bfce6aef2 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:49.202554 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e200e94-e1b4-4a8e-8f2a-306bfce6aef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:49.202823 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e200e94-e1b4-4a8e-8f2a-306bfce6aef2 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:49.203457 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e200e94-e1b4-4a8e-8f2a-306bfce6aef2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:49.204796 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e200e94-e1b4-4a8e-8f2a-306bfce6aef2 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:49.210200 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 402/775] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Sep 12 14:00:49 2022] GET /identity/v3/services => generated 772 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:49.357242 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc23cf3-3ffe-4d30-968f-a7610eb49293 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:49.463151 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:49.472044 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc23cf3-3ffe-4d30-968f-a7610eb49293 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:49.521290 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021f45a-c1de-4570-9309-1605103c832e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:49.521975 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021f45a-c1de-4570-9309-1605103c832e None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:49.522257 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021f45a-c1de-4570-9309-1605103c832e None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:49.522529 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021f45a-c1de-4570-9309-1605103c832e None admin] PATH_INFO: `/v3/services/network` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:49.523264 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b021f45a-c1de-4570-9309-1605103c832e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:49.525071 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b021f45a-c1de-4570-9309-1605103c832e None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:49.534143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-b021f45a-c1de-4570-9309-1605103c832e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 14:00:49.534143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:49.534143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 14:00:49.534143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 14:00:49.534143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 14:00:49.534143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 14:00:49.534143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 14:00:49.534143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:00:49.534143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 14:00:49.534143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:49.534143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 14:00:49.534143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:49.534143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:49.534143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:49.534143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:49.534143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:49.534143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:49.535159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:49.535159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:49.535159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:49.535159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:49.535159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:49.535159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:49.535159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 14:00:49.535159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 14:00:49.535159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 14:00:49.535159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 14:00:49.535159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:49.535159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:49.535159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:49.535159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:49.535159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:49.535159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:49.536267 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:49.536267 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:49.536267 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:49.536267 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:49.536267 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:49.536267 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:49.536267 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:49.536267 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:49.536267 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:49.536267 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:49.536267 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 14:00:49.536267 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:00:49.536267 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 14:00:49.536267 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:49.536267 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 374/776] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:00:49 2022] GET /identity/v3/services/network => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:49.543380 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e200e94-e1b4-4a8e-8f2a-306bfce6aef2 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:49.571270 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:49.652621 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e200e94-e1b4-4a8e-8f2a-306bfce6aef2 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:49.696214 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3038f41-9b8e-4153-a089-b22014ba370a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:49.697094 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3038f41-9b8e-4153-a089-b22014ba370a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:49.697369 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3038f41-9b8e-4153-a089-b22014ba370a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:49.697690 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3038f41-9b8e-4153-a089-b22014ba370a None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:49.698421 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3038f41-9b8e-4153-a089-b22014ba370a None admin] RBAC: Adding query filter params (name=network) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:49.698763 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3038f41-9b8e-4153-a089-b22014ba370a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:49.700206 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3038f41-9b8e-4153-a089-b22014ba370a None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:49.705625 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 403/777] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Sep 12 14:00:49 2022] GET /identity/v3/services?name=network => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:49.712239 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021f45a-c1de-4570-9309-1605103c832e None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:49.826879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021f45a-c1de-4570-9309-1605103c832e None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:49.875635 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb44e849-b7d8-4401-b4dc-752ca6617a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:49.876263 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb44e849-b7d8-4401-b4dc-752ca6617a3c None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:49.876542 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb44e849-b7d8-4401-b4dc-752ca6617a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:49.876797 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb44e849-b7d8-4401-b4dc-752ca6617a3c None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:49.877446 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb44e849-b7d8-4401-b4dc-752ca6617a3c None admin] RBAC: Adding query filter params (type=network) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:49.877747 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb44e849-b7d8-4401-b4dc-752ca6617a3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:49.879156 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb44e849-b7d8-4401-b4dc-752ca6617a3c None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:49.887062 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 375/778] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Sep 12 14:00:49 2022] GET /identity/v3/services?type=network => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:49.891939 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3038f41-9b8e-4153-a089-b22014ba370a None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:50.011937 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:50.031140 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3038f41-9b8e-4153-a089-b22014ba370a None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:50.075415 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c9ccc-f8dc-41e0-844a-bc2895a53678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:50.076025 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c9ccc-f8dc-41e0-844a-bc2895a53678 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:50.076299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c9ccc-f8dc-41e0-844a-bc2895a53678 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:50.076562 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c9ccc-f8dc-41e0-844a-bc2895a53678 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:50.077308 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c9ccc-f8dc-41e0-844a-bc2895a53678 None admin] RBAC: Adding query filter params (interface=public, service_id=105da78160ba44638745a3db3f3e5b88, region_id=RegionOne) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:50.077552 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c9ccc-f8dc-41e0-844a-bc2895a53678 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:50.078899 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c9ccc-f8dc-41e0-844a-bc2895a53678 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:50.086928 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 404/779] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Sep 12 14:00:49 2022] GET /identity/v3/endpoints?service_id=105da78160ba44638745a3db3f3e5b88&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:50.092788 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb44e849-b7d8-4401-b4dc-752ca6617a3c None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:50.122792 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:50.229022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb44e849-b7d8-4401-b4dc-752ca6617a3c None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:50.263297 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14289891-bf4f-4fa9-a249-5dc88cf4ff8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:50.263903 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14289891-bf4f-4fa9-a249-5dc88cf4ff8a None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:50.264182 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14289891-bf4f-4fa9-a249-5dc88cf4ff8a None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:50.264455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14289891-bf4f-4fa9-a249-5dc88cf4ff8a None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:50.265060 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14289891-bf4f-4fa9-a249-5dc88cf4ff8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:50.266528 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14289891-bf4f-4fa9-a249-5dc88cf4ff8a None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:50.278867 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 376/780] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Sep 12 14:00:50 2022] GET /identity/v3/services => generated 772 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:50.495649 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3457fb19-43c9-4fde-b357-6f6fcd4c8818 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:50.495984 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3457fb19-43c9-4fde-b357-6f6fcd4c8818 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:50.496161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3457fb19-43c9-4fde-b357-6f6fcd4c8818 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:50.497797 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 405/781] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:50.503016 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25c683-1663-4d80-98be-322176f66f52 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:50.503563 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25c683-1663-4d80-98be-322176f66f52 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:50.504030 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25c683-1663-4d80-98be-322176f66f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:50.523836 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-2e25c683-1663-4d80-98be-322176f66f52 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:50.525327 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-2e25c683-1663-4d80-98be-322176f66f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:50.598382 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 377/782] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:00:50.899371 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3457fb19-43c9-4fde-b357-6f6fcd4c8818 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:51.004710 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22880f-1885-489c-88c5-79e02a166ca5 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:51.005221 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22880f-1885-489c-88c5-79e02a166ca5 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:51.005298 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22880f-1885-489c-88c5-79e02a166ca5 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:51.006768 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 378/783] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:51.016572 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9cb1e-b2dd-40d1-af6b-6f33511fecd5 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:51.016849 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9cb1e-b2dd-40d1-af6b-6f33511fecd5 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:51.017118 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9cb1e-b2dd-40d1-af6b-6f33511fecd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:51.040239 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-68c9cb1e-b2dd-40d1-af6b-6f33511fecd5 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:51.041001 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3457fb19-43c9-4fde-b357-6f6fcd4c8818 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:51.041998 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-68c9cb1e-b2dd-40d1-af6b-6f33511fecd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:51.055391 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:51.119827 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 379/784] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 14:00:51.122869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a601708a-0b58-4588-bc31-0b93717c7e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:51.122869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601708a-0b58-4588-bc31-0b93717c7e54 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:51.122869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601708a-0b58-4588-bc31-0b93717c7e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:51.123140 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601708a-0b58-4588-bc31-0b93717c7e54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:51.147355 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a601708a-0b58-4588-bc31-0b93717c7e54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:51.153070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-a601708a-0b58-4588-bc31-0b93717c7e54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:00:51.153070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:51.153070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:51.153070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:51.153070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:51.153070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:51.153070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:51.153070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:51.153070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:51.153070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:51.153070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:51.153070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:51.153070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:51.153070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:51.153070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:51.153070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:51.153984 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:51.153984 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:51.153984 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:51.153984 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:51.153984 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:51.153984 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:51.153984 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:51.153984 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:51.153984 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:51.153984 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:51.153984 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:51.153984 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:51.153984 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:51.153984 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:51.153984 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:51.153984 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:51.154823 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:51.154823 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:51.154823 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:51.154823 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:51.154823 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:51.154823 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:51.154823 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:51.154823 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:51.154823 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:51.154823 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:51.154823 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:00:51.154823 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:51.157357 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 406/785] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 12 14:00:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:51.163298 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9cb1e-b2dd-40d1-af6b-6f33511fecd5 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:51.281396 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9cb1e-b2dd-40d1-af6b-6f33511fecd5 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:51.316523 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29f8c9a-e66b-4f9a-9e60-9067ac7ed88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:51.317573 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f8c9a-e66b-4f9a-9e60-9067ac7ed88f None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:51.318043 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f8c9a-e66b-4f9a-9e60-9067ac7ed88f None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:51.318513 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f8c9a-e66b-4f9a-9e60-9067ac7ed88f None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:51.319394 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29f8c9a-e66b-4f9a-9e60-9067ac7ed88f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:51.319887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29f8c9a-e66b-4f9a-9e60-9067ac7ed88f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:51.322360 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29f8c9a-e66b-4f9a-9e60-9067ac7ed88f None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:51.326678 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c9861-72c2-4332-b48b-19498db403a4 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:51.327049 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c9861-72c2-4332-b48b-19498db403a4 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:51.327386 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c9861-72c2-4332-b48b-19498db403a4 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:51.329076 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 407/786] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:51.336359 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493bcde6-ea4a-4da6-b2cf-982af807cc9b None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:51.336739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493bcde6-ea4a-4da6-b2cf-982af807cc9b None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:51.337102 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493bcde6-ea4a-4da6-b2cf-982af807cc9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:51.337782 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 380/787] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 14:00:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:51.346225 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29f8c9a-e66b-4f9a-9e60-9067ac7ed88f None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:51.362863 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:51.363489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-493bcde6-ea4a-4da6-b2cf-982af807cc9b None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:51.364743 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-493bcde6-ea4a-4da6-b2cf-982af807cc9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:51.369614 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:51.448125 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 408/788] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:00:51.503950 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29f8c9a-e66b-4f9a-9e60-9067ac7ed88f None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:51.541836 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d898bdb-701c-40e9-abb9-4c0e0e2a7bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:51.542487 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d898bdb-701c-40e9-abb9-4c0e0e2a7bdc None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:51.542759 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d898bdb-701c-40e9-abb9-4c0e0e2a7bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:51.543058 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d898bdb-701c-40e9-abb9-4c0e0e2a7bdc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:51.555677 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d898bdb-701c-40e9-abb9-4c0e0e2a7bdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:51.557106 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d898bdb-701c-40e9-abb9-4c0e0e2a7bdc None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:51.572439 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-0d898bdb-701c-40e9-abb9-4c0e0e2a7bdc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:51.572439 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:51.572439 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:51.572439 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:51.572439 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:51.572439 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:51.572439 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:51.572439 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:51.572439 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:51.572439 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:51.572439 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:51.572439 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:51.572439 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:51.572439 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:51.572439 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:51.572439 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:51.573374 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:51.573374 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:51.573374 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:51.573374 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:51.573374 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:51.573374 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:51.573374 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:51.573374 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:51.573374 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:51.573374 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:51.573374 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:51.573374 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:51.573374 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:51.573374 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:51.573374 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:51.573374 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:51.573374 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:51.574288 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:51.574288 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:51.574288 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:51.574288 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 381/789] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:00:51.579435 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493bcde6-ea4a-4da6-b2cf-982af807cc9b None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:51.637092 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d898bdb-701c-40e9-abb9-4c0e0e2a7bdc None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:51.706633 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:51.755653 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d898bdb-701c-40e9-abb9-4c0e0e2a7bdc None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:51.778873 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493bcde6-ea4a-4da6-b2cf-982af807cc9b None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:51.793064 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbab91e-299c-43be-b93e-caf6379aede5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:51.793701 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbab91e-299c-43be-b93e-caf6379aede5 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:51.794028 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbab91e-299c-43be-b93e-caf6379aede5 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:51.794295 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbab91e-299c-43be-b93e-caf6379aede5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:51.805681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbab91e-299c-43be-b93e-caf6379aede5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:51.811541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-3bbab91e-299c-43be-b93e-caf6379aede5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:00:51.811541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:51.811541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:51.811541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:51.811541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:51.811541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:51.811541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:51.811541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:51.811541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:51.811541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:51.811541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:51.811541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:51.811541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:51.811541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:51.811541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:51.811541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:51.812512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:51.812512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:51.812512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:51.812512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:51.812512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:51.812512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:51.812512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:51.812512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:51.812512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:51.812512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:51.812512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:51.812512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:51.812512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:51.812512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:51.812512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:51.812512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:51.813363 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:51.813363 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:51.813363 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:51.813363 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:51.813363 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:51.813363 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:51.813363 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:51.813363 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:51.813363 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:51.813363 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:51.813363 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:00:51.813363 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:51.813363 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 382/790] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 12 14:00:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:00:51.814071 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbab91e-299c-43be-b93e-caf6379aede5 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:51.826219 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d5ff84-5ad7-4188-a8ee-1bd6558ab68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:51.827222 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5ff84-5ad7-4188-a8ee-1bd6558ab68a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:51.827778 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5ff84-5ad7-4188-a8ee-1bd6558ab68a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:51.828249 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5ff84-5ad7-4188-a8ee-1bd6558ab68a None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:51.829095 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d5ff84-5ad7-4188-a8ee-1bd6558ab68a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:51.829595 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d5ff84-5ad7-4188-a8ee-1bd6558ab68a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:51.831326 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d5ff84-5ad7-4188-a8ee-1bd6558ab68a None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:51.842192 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:51.850300 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 409/791] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:51.850300 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d5ff84-5ad7-4188-a8ee-1bd6558ab68a None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:51.932330 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbab91e-299c-43be-b93e-caf6379aede5 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:51.955760 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:51.957262 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d5ff84-5ad7-4188-a8ee-1bd6558ab68a None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:51.977823 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2673920c-4e4f-4040-ab6f-c3f50433aeaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:51.979491 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673920c-4e4f-4040-ab6f-c3f50433aeaf None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:51.980128 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673920c-4e4f-4040-ab6f-c3f50433aeaf None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:51.980752 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673920c-4e4f-4040-ab6f-c3f50433aeaf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:51.998175 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2673920c-4e4f-4040-ab6f-c3f50433aeaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:52.000627 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d2136-3107-4aae-8545-486f2cbd7c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:52.001388 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d2136-3107-4aae-8545-486f2cbd7c00 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:52.001857 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d2136-3107-4aae-8545-486f2cbd7c00 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:52.002263 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d2136-3107-4aae-8545-486f2cbd7c00 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:52.003128 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701d2136-3107-4aae-8545-486f2cbd7c00 None admin] RBAC: Adding query filter params (name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:52.003565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701d2136-3107-4aae-8545-486f2cbd7c00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:52.004915 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-2673920c-4e4f-4040-ab6f-c3f50433aeaf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:00:52.004915 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:52.004915 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:52.004915 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:52.004915 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:52.004915 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:52.004915 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:52.004915 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:52.004915 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:52.004915 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:52.004915 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:52.004915 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:52.004915 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:52.004915 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:52.004915 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:52.004915 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:52.006159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:52.006159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:52.006159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:52.006159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:52.006159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:52.006159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:52.006159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:52.006159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:52.006159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:52.006159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:52.006159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:52.006159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:52.006159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:52.006159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:52.006159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:52.006159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:52.007020 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:52.007020 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:52.007020 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:52.007020 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:52.007020 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:52.007020 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:52.007020 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:52.007020 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:52.007020 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:52.007020 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:52.007020 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:00:52.007020 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:52.008917 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701d2136-3107-4aae-8545-486f2cbd7c00 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:52.011004 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 383/792] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 12 14:00:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:52.012752 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2673920c-4e4f-4040-ab6f-c3f50433aeaf None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:52.021409 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 410/793] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 12 14:00:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:52.022590 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d2136-3107-4aae-8545-486f2cbd7c00 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:52.115351 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:52.134677 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2673920c-4e4f-4040-ab6f-c3f50433aeaf None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:52.142458 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d2136-3107-4aae-8545-486f2cbd7c00 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:52.171558 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb9f64-5a46-4d23-a939-bf7683a97c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:52.172340 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb9f64-5a46-4d23-a939-bf7683a97c16 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:52.172704 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb9f64-5a46-4d23-a939-bf7683a97c16 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:52.173062 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb9f64-5a46-4d23-a939-bf7683a97c16 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:52.180939 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ca023-6692-446b-abfc-b0e39703ae5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:52.181597 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ca023-6692-446b-abfc-b0e39703ae5a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:52.181815 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ca023-6692-446b-abfc-b0e39703ae5a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:52.182085 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ca023-6692-446b-abfc-b0e39703ae5a None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:52.182192 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb9f64-5a46-4d23-a939-bf7683a97c16 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:52.182742 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66ca023-6692-446b-abfc-b0e39703ae5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:52.183117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66ca023-6692-446b-abfc-b0e39703ae5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:52.183650 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb9f64-5a46-4d23-a939-bf7683a97c16 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:52.184503 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66ca023-6692-446b-abfc-b0e39703ae5a None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:52.192796 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 411/794] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 14:00:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:52.193821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ca023-6692-446b-abfc-b0e39703ae5a None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:52.195330 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-b3bb9f64-5a46-4d23-a939-bf7683a97c16 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 12 14:00:52.195330 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:52.195330 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:52.195330 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:52.195330 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:52.195330 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:52.195330 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:52.195330 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:52.195330 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:52.195330 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:52.195330 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:52.195330 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:52.195330 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:00:52.195330 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:00:52.195330 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:00:52.195330 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:00:52.196237 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:52.196237 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:52.196237 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:00:52.196237 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:52.196237 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:00:52.196237 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:52.196237 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:52.196237 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:52.196237 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:52.196237 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:52.196237 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:52.196237 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:52.196237 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:52.196237 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:52.196237 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:52.196237 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:52.196237 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:52.196237 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:52.197440 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:52.197440 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:52.197440 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:00:52.197440 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:00:52.197440 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:00:52.197440 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:00:52.197440 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:00:52.197440 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:00:52.197440 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 12 14:00:52.197440 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:52.201201 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 384/795] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 12 14:00:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:52.202439 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848d3a1-dc00-469f-8406-c9e8a76cf169 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:52.202730 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848d3a1-dc00-469f-8406-c9e8a76cf169 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:52.203025 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848d3a1-dc00-469f-8406-c9e8a76cf169 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:52.206991 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 385/796] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:52.206991 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848d3a1-dc00-469f-8406-c9e8a76cf169 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:52.254253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:52.304999 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ca023-6692-446b-abfc-b0e39703ae5a None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:52.317571 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848d3a1-dc00-469f-8406-c9e8a76cf169 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:52.329181 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:52.341182 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06e662c-0e11-471e-ba46-34c6c9fd7e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:52.342030 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e662c-0e11-471e-ba46-34c6c9fd7e31 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:52.342311 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e662c-0e11-471e-ba46-34c6c9fd7e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:52.342585 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e662c-0e11-471e-ba46-34c6c9fd7e31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:52.353447 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bd5618-4d6b-4d00-9800-244dbffec7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:52.353991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06e662c-0e11-471e-ba46-34c6c9fd7e31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:52.354113 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd5618-4d6b-4d00-9800-244dbffec7dd None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:52.354327 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd5618-4d6b-4d00-9800-244dbffec7dd None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:52.354605 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd5618-4d6b-4d00-9800-244dbffec7dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:52.355463 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06e662c-0e11-471e-ba46-34c6c9fd7e31 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:52.365783 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bd5618-4d6b-4d00-9800-244dbffec7dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:52.367873 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bd5618-4d6b-4d00-9800-244dbffec7dd None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:52.371066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-f06e662c-0e11-471e-ba46-34c6c9fd7e31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:52.371066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:52.371066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:52.371066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:52.371066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:52.371066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:52.371066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:52.371066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:52.371066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:52.371066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:52.371066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:52.371066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:52.371066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:52.371066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:52.371066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:52.371066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:52.371980 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:52.371980 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:52.371980 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:52.371980 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:52.371980 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:52.371980 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:52.371980 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:52.371980 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:52.371980 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:52.371980 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:52.371980 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:52.371980 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:52.371980 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:52.371980 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:52.371980 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:52.371980 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:52.371980 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:52.372886 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:52.372886 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:52.372886 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:52.372886 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 412/797] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:52.373526 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06e662c-0e11-471e-ba46-34c6c9fd7e31 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:52.387346 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-20bd5618-4d6b-4d00-9800-244dbffec7dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:52.387346 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:52.387346 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:52.387346 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:52.387346 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:52.387346 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:52.387346 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:52.387346 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:52.387346 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:52.387346 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:52.387346 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:52.387346 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:52.387346 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:52.387346 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:52.387346 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:52.387346 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:52.388324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:52.388324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:52.388324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:52.388324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:52.388324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:52.388324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:52.388324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:52.388324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:52.388324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:52.388324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:52.388324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:52.388324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:52.388324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:52.388324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:52.388324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:52.388324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:52.388324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:52.389223 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:52.389223 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:52.389223 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:52.389223 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 386/798] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:52.389854 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51752abe-039b-4658-a664-bb1372ff923b None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:52.389854 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51752abe-039b-4658-a664-bb1372ff923b None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:52.390093 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51752abe-039b-4658-a664-bb1372ff923b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:52.413968 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-51752abe-039b-4658-a664-bb1372ff923b None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:52.415339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-51752abe-039b-4658-a664-bb1372ff923b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:52.490634 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 387/799] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:00:52.491513 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51752abe-039b-4658-a664-bb1372ff923b None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:52.506831 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06e662c-0e11-471e-ba46-34c6c9fd7e31 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:52.520001 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:52.546436 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265fa41-aa69-4c13-bd51-150ad2ce7b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:52.547134 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265fa41-aa69-4c13-bd51-150ad2ce7b27 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:52.547418 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265fa41-aa69-4c13-bd51-150ad2ce7b27 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:52.547718 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265fa41-aa69-4c13-bd51-150ad2ce7b27 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:52.548419 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7265fa41-aa69-4c13-bd51-150ad2ce7b27 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:52.548722 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7265fa41-aa69-4c13-bd51-150ad2ce7b27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:52.550132 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7265fa41-aa69-4c13-bd51-150ad2ce7b27 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:52.573720 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 413/800] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Sep 12 14:00:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:52.573720 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265fa41-aa69-4c13-bd51-150ad2ce7b27 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:52.612506 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51752abe-039b-4658-a664-bb1372ff923b None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:52.621730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:52.650974 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e82ed94-897e-4fb3-ad90-fa6dc2bcdc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:52.651614 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82ed94-897e-4fb3-ad90-fa6dc2bcdc2f None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:52.651954 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82ed94-897e-4fb3-ad90-fa6dc2bcdc2f None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:52.652363 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82ed94-897e-4fb3-ad90-fa6dc2bcdc2f None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:52.653024 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e82ed94-897e-4fb3-ad90-fa6dc2bcdc2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:52.653408 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e82ed94-897e-4fb3-ad90-fa6dc2bcdc2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:52.655173 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e82ed94-897e-4fb3-ad90-fa6dc2bcdc2f None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:52.667241 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 388/801] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:52.671501 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e82ed94-897e-4fb3-ad90-fa6dc2bcdc2f None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:52.705546 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265fa41-aa69-4c13-bd51-150ad2ce7b27 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:52.752340 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a68de-4ca7-4ed5-9560-745f034ac842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:52.753293 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a68de-4ca7-4ed5-9560-745f034ac842 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:52.753799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a68de-4ca7-4ed5-9560-745f034ac842 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:52.754219 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a68de-4ca7-4ed5-9560-745f034ac842 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:52.755080 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38a68de-4ca7-4ed5-9560-745f034ac842 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:52.755539 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38a68de-4ca7-4ed5-9560-745f034ac842 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:52.759051 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38a68de-4ca7-4ed5-9560-745f034ac842 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:52.766728 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:52.782932 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 414/802] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:52.783894 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a68de-4ca7-4ed5-9560-745f034ac842 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:52.798481 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e82ed94-897e-4fb3-ad90-fa6dc2bcdc2f None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:52.833350 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aeb732-5e4e-4adb-b351-aac303234589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:52.834053 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aeb732-5e4e-4adb-b351-aac303234589 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:52.834474 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aeb732-5e4e-4adb-b351-aac303234589 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:52.834751 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aeb732-5e4e-4adb-b351-aac303234589 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:52.846362 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aeb732-5e4e-4adb-b351-aac303234589 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:52.847855 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aeb732-5e4e-4adb-b351-aac303234589 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:52.862639 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-91aeb732-5e4e-4adb-b351-aac303234589 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:52.862639 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:52.862639 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:52.862639 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:52.862639 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:52.862639 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:52.862639 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:52.862639 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:52.862639 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:52.862639 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:52.862639 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:52.862639 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:52.862639 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:52.862639 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:52.862639 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:52.862639 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:52.863602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:52.863602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:52.863602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:52.863602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:52.863602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:52.863602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:52.863602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:52.863602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:52.863602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:52.863602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:52.863602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:52.863602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:52.863602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:52.863602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:52.863602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:52.863602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:52.863602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:52.864490 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:52.864490 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:52.864490 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:52.868405 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 389/803] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:52.869346 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aeb732-5e4e-4adb-b351-aac303234589 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:52.891093 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:52.897685 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a68de-4ca7-4ed5-9560-745f034ac842 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:52.931623 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aecd247-f700-4220-96a9-65579ddb8416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:52.932236 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aecd247-f700-4220-96a9-65579ddb8416 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:52.932510 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aecd247-f700-4220-96a9-65579ddb8416 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:52.932778 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aecd247-f700-4220-96a9-65579ddb8416 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:52.940033 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aecd247-f700-4220-96a9-65579ddb8416 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:52.941425 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aecd247-f700-4220-96a9-65579ddb8416 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:52.952962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-7aecd247-f700-4220-96a9-65579ddb8416 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 14:00:52.952962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:52.952962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:52.952962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:52.952962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:52.952962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:52.952962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:52.952962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:52.952962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:52.952962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:52.952962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:52.952962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:52.952962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:00:52.952962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:00:52.952962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:00:52.952962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:00:52.959161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:52.959161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:52.959161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:00:52.959161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:52.959161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:00:52.959161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:52.959161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:52.959161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:52.959161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:52.959161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:52.959161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:52.959161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:52.959161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:52.959161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:52.959161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:52.959161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:52.959161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:52.959161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:52.960345 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:52.960345 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:52.960345 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:00:52.960345 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:00:52.960345 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:00:52.960345 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:00:52.960345 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:00:52.960345 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:00:52.960345 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 14:00:52.960345 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:52.960345 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 415/804] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Sep 12 14:00:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:52.960345 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aecd247-f700-4220-96a9-65579ddb8416 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:52.973521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aeb732-5e4e-4adb-b351-aac303234589 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:53.001851 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:53.006749 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f79340-2efa-44bf-a4cd-7b942e7503b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:53.008325 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f79340-2efa-44bf-a4cd-7b942e7503b1 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:53.008325 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f79340-2efa-44bf-a4cd-7b942e7503b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:53.008500 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f79340-2efa-44bf-a4cd-7b942e7503b1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:53.008997 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f79340-2efa-44bf-a4cd-7b942e7503b1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:53.010391 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f79340-2efa-44bf-a4cd-7b942e7503b1 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:53.018968 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-f1f79340-2efa-44bf-a4cd-7b942e7503b1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 14:00:53.018968 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:53.018968 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 14:00:53.018968 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 14:00:53.018968 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 14:00:53.018968 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 14:00:53.018968 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 14:00:53.018968 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:00:53.018968 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 14:00:53.018968 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:53.018968 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 14:00:53.018968 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:53.018968 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:53.018968 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:53.018968 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:53.018968 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:53.018968 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:53.019982 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:53.019982 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:53.019982 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:53.019982 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:53.019982 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:53.019982 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:53.019982 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 14:00:53.019982 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 14:00:53.019982 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 14:00:53.019982 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 14:00:53.019982 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:53.019982 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:53.019982 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:53.019982 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:53.019982 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:53.019982 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:53.020822 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:53.020822 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:53.020822 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:53.020822 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:53.020822 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:53.020822 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:53.020822 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:53.020822 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:53.020822 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:53.020822 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:53.020822 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 14:00:53.020822 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:00:53.020822 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 14:00:53.020822 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:53.020822 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 390/805] 10.222.0.42 () {56 vars in 1129 bytes} [Mon Sep 12 14:00:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:53.021613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f79340-2efa-44bf-a4cd-7b942e7503b1 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:53.056635 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aecd247-f700-4220-96a9-65579ddb8416 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:53.091545 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db982e0-6679-45bf-8936-818d469a942b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:53.092166 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db982e0-6679-45bf-8936-818d469a942b None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:53.092442 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db982e0-6679-45bf-8936-818d469a942b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:53.092707 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db982e0-6679-45bf-8936-818d469a942b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:53.100064 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db982e0-6679-45bf-8936-818d469a942b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:53.102079 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db982e0-6679-45bf-8936-818d469a942b None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:53.113161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-7db982e0-6679-45bf-8936-818d469a942b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 14:00:53.113161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:53.113161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:53.113161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:53.113161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:53.113161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:53.113161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:53.113161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:53.113161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:53.113161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:53.113161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:53.113161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:53.113161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:00:53.113161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:00:53.113161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:00:53.113161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:00:53.114108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:53.114108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:53.114108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:00:53.114108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:53.114108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:00:53.114108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:53.114108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:53.114108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:53.114108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:53.114108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:53.114108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:53.114108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:53.114108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:53.114108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:53.114108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:53.114108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:53.114108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:53.114108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:53.115092 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:53.115092 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:53.115092 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:00:53.115092 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:00:53.115092 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:00:53.115092 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:00:53.115092 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:00:53.115092 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:00:53.115092 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 14:00:53.115092 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:53.115092 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 416/806] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Sep 12 14:00:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:53.115691 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a4b97-f038-4020-abe0-423d565bf1c1 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:53.115759 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a4b97-f038-4020-abe0-423d565bf1c1 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:53.115976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a4b97-f038-4020-abe0-423d565bf1c1 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:53.117977 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 417/807] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:53.118589 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01a4b97-f038-4020-abe0-423d565bf1c1 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:53.134793 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:53.136646 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f79340-2efa-44bf-a4cd-7b942e7503b1 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:53.170361 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6f331-d8a1-412e-af2c-354414995a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:53.170993 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6f331-d8a1-412e-af2c-354414995a87 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:53.171270 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6f331-d8a1-412e-af2c-354414995a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:53.171542 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6f331-d8a1-412e-af2c-354414995a87 None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:53.172183 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc6f331-d8a1-412e-af2c-354414995a87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:53.172484 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc6f331-d8a1-412e-af2c-354414995a87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:53.173835 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc6f331-d8a1-412e-af2c-354414995a87 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:53.186333 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 391/808] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:53.187140 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6f331-d8a1-412e-af2c-354414995a87 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:53.223927 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01a4b97-f038-4020-abe0-423d565bf1c1 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:53.259681 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc83821-4ffe-45f6-a824-0343717a3c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:53.260298 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc83821-4ffe-45f6-a824-0343717a3c32 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:53.260628 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc83821-4ffe-45f6-a824-0343717a3c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:53.260847 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc83821-4ffe-45f6-a824-0343717a3c32 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:53.261590 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc83821-4ffe-45f6-a824-0343717a3c32 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:53.261892 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc83821-4ffe-45f6-a824-0343717a3c32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:53.263383 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc83821-4ffe-45f6-a824-0343717a3c32 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:53.283075 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 418/809] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Sep 12 14:00:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:53.283942 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc83821-4ffe-45f6-a824-0343717a3c32 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:53.355102 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:53.355312 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:53.389889 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc83821-4ffe-45f6-a824-0343717a3c32 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:53.402547 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6f331-d8a1-412e-af2c-354414995a87 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:53.425280 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a13be8-e304-480e-baff-3475c8509a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:53.425885 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a13be8-e304-480e-baff-3475c8509a4a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:53.426165 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a13be8-e304-480e-baff-3475c8509a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:53.426440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a13be8-e304-480e-baff-3475c8509a4a None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:53.427194 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a13be8-e304-480e-baff-3475c8509a4a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:53.427495 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a13be8-e304-480e-baff-3475c8509a4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:53.428891 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a13be8-e304-480e-baff-3475c8509a4a None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:53.437371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a951a-10a3-4bcd-a01c-c4a249d56f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:53.437968 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a951a-10a3-4bcd-a01c-c4a249d56f4e None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:53.438242 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a951a-10a3-4bcd-a01c-c4a249d56f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:53.438510 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a951a-10a3-4bcd-a01c-c4a249d56f4e None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:53.439166 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a951a-10a3-4bcd-a01c-c4a249d56f4e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:53.439461 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a951a-10a3-4bcd-a01c-c4a249d56f4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:53.440794 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a951a-10a3-4bcd-a01c-c4a249d56f4e None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:53.445059 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 392/810] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Sep 12 14:00:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:53.450810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fdbbd7-806d-44e1-88fe-0587f3f53324 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:53.450992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 419/811] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Sep 12 14:00:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:53.451091 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fdbbd7-806d-44e1-88fe-0587f3f53324 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:53.451091 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fdbbd7-806d-44e1-88fe-0587f3f53324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:53.451244 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a13be8-e304-480e-baff-3475c8509a4a None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:53.465037 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-04fdbbd7-806d-44e1-88fe-0587f3f53324 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:53.466208 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-04fdbbd7-806d-44e1-88fe-0587f3f53324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:53.531949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 393/812] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:00:53.532696 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fdbbd7-806d-44e1-88fe-0587f3f53324 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:53.546581 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a13be8-e304-480e-baff-3475c8509a4a None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:53.575205 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:53.581264 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2985db-7f81-44d6-bf6b-099145633944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:53.581901 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2985db-7f81-44d6-bf6b-099145633944 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:53.582181 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2985db-7f81-44d6-bf6b-099145633944 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:53.582454 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2985db-7f81-44d6-bf6b-099145633944 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:53.589526 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:53.589647 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2985db-7f81-44d6-bf6b-099145633944 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:53.591076 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2985db-7f81-44d6-bf6b-099145633944 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:53.603266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-af2985db-7f81-44d6-bf6b-099145633944 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:53.603266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:53.603266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:53.603266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:53.603266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:53.603266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:53.603266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:53.603266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:53.603266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:53.603266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:53.603266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:53.603266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:53.603266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:00:53.603266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:00:53.603266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:00:53.604172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:00:53.604172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:53.604172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:53.604172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:53.604172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:53.604172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:53.604172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:53.604172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:53.604172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:53.604172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:53.604172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:53.604172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:53.604172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:53.604172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:53.604172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:53.604172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:53.604172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:53.609871 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:00:53.609871 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:00:53.609871 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:00:53.609871 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:00:53.609871 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:00:53.609871 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:00:53.609871 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:53.609871 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:53.609871 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 420/813] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:00:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:53.609871 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2985db-7f81-44d6-bf6b-099145633944 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:53.646878 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fdbbd7-806d-44e1-88fe-0587f3f53324 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:53.682454 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee27d632-6301-4ac3-9f4b-baa3cdd3348a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:53.683106 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27d632-6301-4ac3-9f4b-baa3cdd3348a None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:53.683380 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27d632-6301-4ac3-9f4b-baa3cdd3348a None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:53.683644 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27d632-6301-4ac3-9f4b-baa3cdd3348a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:53.701266 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee27d632-6301-4ac3-9f4b-baa3cdd3348a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:53.702691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee27d632-6301-4ac3-9f4b-baa3cdd3348a None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:53.714489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2985db-7f81-44d6-bf6b-099145633944 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:53.719648 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-ee27d632-6301-4ac3-9f4b-baa3cdd3348a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:53.719648 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:53.719648 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:53.719648 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:53.719648 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:53.719648 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:53.719648 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:53.719648 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:53.719648 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:53.719648 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:53.719648 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:53.719648 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:53.719648 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:53.719648 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:53.719648 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:53.719648 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:53.720562 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:53.720562 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:53.720562 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:53.720562 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:53.720562 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:53.720562 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:53.720562 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:53.720562 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:53.720562 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:53.720562 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:53.720562 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:53.720562 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:53.720562 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:53.720562 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:53.720562 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:53.720562 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:53.720562 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:53.721437 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:53.721437 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:53.721437 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:53.722187 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 394/814] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:53.723048 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee27d632-6301-4ac3-9f4b-baa3cdd3348a None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:53.754689 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f13b1f-51ba-4402-a47d-b4a82da6c8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:53.755319 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f13b1f-51ba-4402-a47d-b4a82da6c8f4 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:53.755605 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f13b1f-51ba-4402-a47d-b4a82da6c8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:53.755863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f13b1f-51ba-4402-a47d-b4a82da6c8f4 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:53.756481 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f13b1f-51ba-4402-a47d-b4a82da6c8f4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:53.756776 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f13b1f-51ba-4402-a47d-b4a82da6c8f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:53.758812 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f13b1f-51ba-4402-a47d-b4a82da6c8f4 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:53.764865 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 421/815] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Sep 12 14:00:53 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:53.767699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f13b1f-51ba-4402-a47d-b4a82da6c8f4 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:53.825014 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:53.829025 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee27d632-6301-4ac3-9f4b-baa3cdd3348a None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:53.830714 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:53.866105 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a24ad-f546-493e-8c2b-b018719e7c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:53.866734 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a24ad-f546-493e-8c2b-b018719e7c69 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:53.867039 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a24ad-f546-493e-8c2b-b018719e7c69 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:53.867309 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a24ad-f546-493e-8c2b-b018719e7c69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:53.869841 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f13b1f-51ba-4402-a47d-b4a82da6c8f4 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:53.904600 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cebb6e-39a2-49e5-bcf4-25395daca192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:53.905197 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cebb6e-39a2-49e5-bcf4-25395daca192 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:53.905474 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cebb6e-39a2-49e5-bcf4-25395daca192 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:53.905806 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cebb6e-39a2-49e5-bcf4-25395daca192 None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:53.906494 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cebb6e-39a2-49e5-bcf4-25395daca192 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:53.906795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cebb6e-39a2-49e5-bcf4-25395daca192 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:53.908237 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cebb6e-39a2-49e5-bcf4-25395daca192 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:53.920201 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 422/816] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 12 14:00:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:53.920976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cebb6e-39a2-49e5-bcf4-25395daca192 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:53.933659 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17a24ad-f546-493e-8c2b-b018719e7c69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:53.935131 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17a24ad-f546-493e-8c2b-b018719e7c69 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:54.016818 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cebb6e-39a2-49e5-bcf4-25395daca192 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:54.030749 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:54.033960 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 395/817] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:00:53 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 14:00:54.034855 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a24ad-f546-493e-8c2b-b018719e7c69 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:54.040397 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:54.052712 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3120a-b63d-4412-9451-297e40b1c056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:54.053320 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3120a-b63d-4412-9451-297e40b1c056 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:54.053598 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3120a-b63d-4412-9451-297e40b1c056 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:54.053872 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3120a-b63d-4412-9451-297e40b1c056 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:54.055049 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e3120a-b63d-4412-9451-297e40b1c056 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:54.055351 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e3120a-b63d-4412-9451-297e40b1c056 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:54.056699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e3120a-b63d-4412-9451-297e40b1c056 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:54.067531 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 423/818] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:54.068309 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3120a-b63d-4412-9451-297e40b1c056 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:54.136102 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a24ad-f546-493e-8c2b-b018719e7c69 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:54.167734 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efd02d-64a2-4b2f-a70e-ed73455a8871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:54.168318 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efd02d-64a2-4b2f-a70e-ed73455a8871 None admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:54.168589 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efd02d-64a2-4b2f-a70e-ed73455a8871 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:54.168854 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efd02d-64a2-4b2f-a70e-ed73455a8871 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:54.169507 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5efd02d-64a2-4b2f-a70e-ed73455a8871 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:54.170876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5efd02d-64a2-4b2f-a70e-ed73455a8871 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:54.173251 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3120a-b63d-4412-9451-297e40b1c056 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:54.191701 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 396/819] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:00:54 2022] POST /identity/v3/endpoints => generated 346 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:00:54.192604 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efd02d-64a2-4b2f-a70e-ed73455a8871 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:54.215734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc24102d-6aac-4d7a-b530-faa717e0a554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:54.216381 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24102d-6aac-4d7a-b530-faa717e0a554 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:54.216657 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24102d-6aac-4d7a-b530-faa717e0a554 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:54.216923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24102d-6aac-4d7a-b530-faa717e0a554 None admin] PATH_INFO: `/v3/services/network` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:54.217508 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc24102d-6aac-4d7a-b530-faa717e0a554 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:54.218923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc24102d-6aac-4d7a-b530-faa717e0a554 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:54.231467 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-cc24102d-6aac-4d7a-b530-faa717e0a554 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 14:00:54.231467 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:54.231467 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 14:00:54.231467 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 14:00:54.231467 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 14:00:54.231467 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 14:00:54.231467 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 14:00:54.231467 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:00:54.231467 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 14:00:54.231467 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:54.231467 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 14:00:54.231467 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:54.231467 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:54.231467 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:54.231467 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:54.231467 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:54.231467 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:54.232644 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:54.232644 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:54.232644 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:54.232644 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:54.232644 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:54.232644 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:54.232644 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 14:00:54.232644 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 14:00:54.232644 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 14:00:54.232644 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 14:00:54.232644 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:54.232644 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:54.232644 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:54.232644 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:54.232644 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:54.232644 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:54.233538 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:54.233538 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:54.233538 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:54.233538 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:54.233538 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:54.233538 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:54.233538 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:54.233538 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:54.233538 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:54.233538 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:54.233538 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 14:00:54.233538 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:00:54.233538 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 14:00:54.233538 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:54.233538 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 424/820] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:00:54 2022] GET /identity/v3/services/network => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:54.235557 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc24102d-6aac-4d7a-b530-faa717e0a554 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:54.273749 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:54.286800 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:54.336973 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efd02d-64a2-4b2f-a70e-ed73455a8871 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:54.349726 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc24102d-6aac-4d7a-b530-faa717e0a554 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:54.374790 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd224300-d1d3-4642-bb9f-41b49d3c06fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:54.375475 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd224300-d1d3-4642-bb9f-41b49d3c06fd None admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:54.375749 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd224300-d1d3-4642-bb9f-41b49d3c06fd None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:54.376018 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd224300-d1d3-4642-bb9f-41b49d3c06fd None admin] PATH_INFO: `/v3/projects/f666a617f8d14eb18bbeb076032b82f2/users/25a87ea9c3c5409b97a35c4857ae3ca6/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:54.378319 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd224300-d1d3-4642-bb9f-41b49d3c06fd None admin] RBAC: Authorizing `identity:create_grant(project_id=f666a617f8d14eb18bbeb076032b82f2, user_id=25a87ea9c3c5409b97a35c4857ae3ca6, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:54.380056 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd224300-d1d3-4642-bb9f-41b49d3c06fd None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:54.391679 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c962744f-4b01-4a76-8a6f-b2850243a5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:54.392982 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962744f-4b01-4a76-8a6f-b2850243a5be None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:54.392982 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962744f-4b01-4a76-8a6f-b2850243a5be None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:54.392982 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962744f-4b01-4a76-8a6f-b2850243a5be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:54.393811 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 397/821] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 12 14:00:54 2022] PUT /identity/v3/projects/f666a617f8d14eb18bbeb076032b82f2/users/25a87ea9c3c5409b97a35c4857ae3ca6/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:00:54.394416 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd224300-d1d3-4642-bb9f-41b49d3c06fd None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:54.409346 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c962744f-4b01-4a76-8a6f-b2850243a5be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:54.410824 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c962744f-4b01-4a76-8a6f-b2850243a5be None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:54.431869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-c962744f-4b01-4a76-8a6f-b2850243a5be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:54.431869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:54.431869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:54.431869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:54.431869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:54.431869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:54.431869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:54.431869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:54.431869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:54.431869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:54.431869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:54.431869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:54.431869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:54.431869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:54.431869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:54.431869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:54.432921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:54.432921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:54.432921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:54.432921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:54.432921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:54.432921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:54.432921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:54.432921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:54.432921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:54.432921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:54.432921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:54.432921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:54.432921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:54.432921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:54.432921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:54.432921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:54.432921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:54.434668 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:54.434668 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:54.434668 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:54.434668 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 425/822] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:54.434668 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c962744f-4b01-4a76-8a6f-b2850243a5be None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:54.538062 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd224300-d1d3-4642-bb9f-41b49d3c06fd None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:54.557438 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:54.560884 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:54.586610 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85116408-a678-431d-bade-0ac8ca97c221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:54.587581 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85116408-a678-431d-bade-0ac8ca97c221 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:54.588039 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85116408-a678-431d-bade-0ac8ca97c221 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:54.588508 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85116408-a678-431d-bade-0ac8ca97c221 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:54.589747 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c962744f-4b01-4a76-8a6f-b2850243a5be None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:54.596073 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85116408-a678-431d-bade-0ac8ca97c221 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:54.597604 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85116408-a678-431d-bade-0ac8ca97c221 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:54.611290 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-85116408-a678-431d-bade-0ac8ca97c221 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:54.611290 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:54.611290 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:54.611290 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:54.611290 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:54.611290 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:54.611290 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:54.611290 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:54.611290 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:54.611290 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:54.611290 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:54.611290 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:54.611290 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:00:54.611290 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:00:54.611290 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:00:54.612400 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:00:54.612400 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:54.612400 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:54.612400 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:54.612400 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:54.612400 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:54.612400 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:54.612400 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:54.612400 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:54.612400 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:54.612400 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:54.612400 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:54.612400 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:54.612400 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:54.612400 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:54.612400 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:54.612400 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:54.613529 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:00:54.613529 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:00:54.613529 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:00:54.613529 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:00:54.613529 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:00:54.613529 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:00:54.613529 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:54.613529 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:54.613529 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 398/823] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:00:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:54.613529 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85116408-a678-431d-bade-0ac8ca97c221 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:54.631861 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d9513-eebc-4ec3-b98a-d032d8bf64fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:54.632250 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d9513-eebc-4ec3-b98a-d032d8bf64fa None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:54.633168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d9513-eebc-4ec3-b98a-d032d8bf64fa None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:54.633442 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d9513-eebc-4ec3-b98a-d032d8bf64fa None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:54.634149 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33d9513-eebc-4ec3-b98a-d032d8bf64fa None admin] RBAC: Adding query filter params (name=network) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:54.634667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33d9513-eebc-4ec3-b98a-d032d8bf64fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:54.636434 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33d9513-eebc-4ec3-b98a-d032d8bf64fa None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:54.642204 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 426/824] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Sep 12 14:00:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:54.642951 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d9513-eebc-4ec3-b98a-d032d8bf64fa None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:54.737892 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85116408-a678-431d-bade-0ac8ca97c221 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:54.770011 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d9513-eebc-4ec3-b98a-d032d8bf64fa None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:54.784280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643e48c0-fd36-43f7-be61-ddd0e2abc525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:54.785588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e48c0-fd36-43f7-be61-ddd0e2abc525 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:54.786028 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e48c0-fd36-43f7-be61-ddd0e2abc525 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:54.786441 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e48c0-fd36-43f7-be61-ddd0e2abc525 None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:54.787513 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643e48c0-fd36-43f7-be61-ddd0e2abc525 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:54.787999 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643e48c0-fd36-43f7-be61-ddd0e2abc525 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:54.790673 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643e48c0-fd36-43f7-be61-ddd0e2abc525 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:54.807287 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 399/825] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:54.808546 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643e48c0-fd36-43f7-be61-ddd0e2abc525 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:54.808681 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a4066-05cc-43e7-a133-349a9de0cbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:54.809445 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a4066-05cc-43e7-a133-349a9de0cbcd None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:54.809816 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a4066-05cc-43e7-a133-349a9de0cbcd None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:54.810187 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a4066-05cc-43e7-a133-349a9de0cbcd None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:54.811095 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782a4066-05cc-43e7-a133-349a9de0cbcd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:54.811668 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782a4066-05cc-43e7-a133-349a9de0cbcd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:54.813962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782a4066-05cc-43e7-a133-349a9de0cbcd None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:54.827722 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 427/826] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 12 14:00:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:54.828768 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a4066-05cc-43e7-a133-349a9de0cbcd None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:54.843529 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:54.866921 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:54.950377 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a4066-05cc-43e7-a133-349a9de0cbcd None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:54.954130 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643e48c0-fd36-43f7-be61-ddd0e2abc525 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:54.991059 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128b98a-7051-4b05-b7c4-315a46334c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:54.992421 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128b98a-7051-4b05-b7c4-315a46334c65 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:54.992791 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128b98a-7051-4b05-b7c4-315a46334c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:54.993199 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128b98a-7051-4b05-b7c4-315a46334c65 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:54.999877 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d22d6-bbdf-4d78-87ca-977c5bc6da3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:55.000768 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d22d6-bbdf-4d78-87ca-977c5bc6da3e None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:55.000887 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9128b98a-7051-4b05-b7c4-315a46334c65 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:55.001287 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d22d6-bbdf-4d78-87ca-977c5bc6da3e None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:55.001715 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d22d6-bbdf-4d78-87ca-977c5bc6da3e None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:55.002497 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d22d6-bbdf-4d78-87ca-977c5bc6da3e None admin] RBAC: Adding query filter params (type=network) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:55.002656 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9128b98a-7051-4b05-b7c4-315a46334c65 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:55.003186 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d22d6-bbdf-4d78-87ca-977c5bc6da3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:55.005058 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d22d6-bbdf-4d78-87ca-977c5bc6da3e None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:55.012060 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 400/827] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Sep 12 14:00:54 2022] GET /identity/v3/services?type=network => generated 350 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:55.013015 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d22d6-bbdf-4d78-87ca-977c5bc6da3e None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:55.017921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-9128b98a-7051-4b05-b7c4-315a46334c65 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:55.017921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:55.017921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:55.017921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:55.017921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:55.017921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:55.017921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:55.017921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:55.017921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:55.017921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:55.017921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:55.017921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:55.017921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:00:55.017921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:00:55.017921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:00:55.019353 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:00:55.019353 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:55.019353 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:55.019353 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:55.019353 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:55.019353 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:55.019353 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:55.019353 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:55.019353 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:55.019353 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:55.019353 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:55.019353 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:55.019353 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:55.019353 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:55.019353 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:55.019353 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:55.019353 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:55.020465 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:00:55.020465 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:00:55.020465 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:00:55.020465 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:00:55.020465 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:00:55.020465 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:00:55.020465 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:55.020465 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:55.021868 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 428/828] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:00:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:55.022779 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128b98a-7051-4b05-b7c4-315a46334c65 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:55.101968 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:55.125833 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d22d6-bbdf-4d78-87ca-977c5bc6da3e None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:55.128186 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:55.140305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128b98a-7051-4b05-b7c4-315a46334c65 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:55.168539 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597359f-68f8-4687-9269-dc362bfbeeb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:55.169668 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597359f-68f8-4687-9269-dc362bfbeeb6 None admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:55.170133 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597359f-68f8-4687-9269-dc362bfbeeb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:55.170645 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597359f-68f8-4687-9269-dc362bfbeeb6 None admin] PATH_INFO: `/v3/projects/f666a617f8d14eb18bbeb076032b82f2/users/1405e758a3af4e8ba659f498cd28ace1/roles/1c413f4e78ef45b7a821bb884f9cea07` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:55.181783 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c15d8-21ec-4379-8732-65b4995dac39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:55.182405 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c15d8-21ec-4379-8732-65b4995dac39 None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:55.182673 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c15d8-21ec-4379-8732-65b4995dac39 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:55.183123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c15d8-21ec-4379-8732-65b4995dac39 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:55.183949 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2c15d8-21ec-4379-8732-65b4995dac39 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:55.185311 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2c15d8-21ec-4379-8732-65b4995dac39 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:55.189460 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4597359f-68f8-4687-9269-dc362bfbeeb6 None admin] RBAC: Authorizing `identity:create_grant(project_id=f666a617f8d14eb18bbeb076032b82f2, user_id=1405e758a3af4e8ba659f498cd28ace1, role_id=1c413f4e78ef45b7a821bb884f9cea07)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:55.191427 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4597359f-68f8-4687-9269-dc362bfbeeb6 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:55.209399 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 401/829] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 12 14:00:55 2022] PUT /identity/v3/projects/f666a617f8d14eb18bbeb076032b82f2/users/1405e758a3af4e8ba659f498cd28ace1/roles/1c413f4e78ef45b7a821bb884f9cea07 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:00:55.210227 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597359f-68f8-4687-9269-dc362bfbeeb6 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:55.212214 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 429/830] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:00:55 2022] POST /identity/v3/endpoints => generated 337 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:00:55.340747 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597359f-68f8-4687-9269-dc362bfbeeb6 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:55.399676 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e554fe-516a-47f1-9596-ec7e71217df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:55.400715 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e554fe-516a-47f1-9596-ec7e71217df8 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:55.401191 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e554fe-516a-47f1-9596-ec7e71217df8 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:55.401668 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e554fe-516a-47f1-9596-ec7e71217df8 None admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:55.402665 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e554fe-516a-47f1-9596-ec7e71217df8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:55.403194 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e554fe-516a-47f1-9596-ec7e71217df8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:55.404864 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e554fe-516a-47f1-9596-ec7e71217df8 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:55.423307 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 402/831] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 12 14:00:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:55.429441 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c15d8-21ec-4379-8732-65b4995dac39 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:55.552481 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c15d8-21ec-4379-8732-65b4995dac39 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:55.584202 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:55.595283 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77000bb7-e76c-41c3-aeb4-d41ac6101ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:55.596068 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77000bb7-e76c-41c3-aeb4-d41ac6101ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:55.596578 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77000bb7-e76c-41c3-aeb4-d41ac6101ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:55.596853 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77000bb7-e76c-41c3-aeb4-d41ac6101ac5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:55.597583 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77000bb7-e76c-41c3-aeb4-d41ac6101ac5 None admin] RBAC: Adding query filter params (role.id=e14564f00daf48a28d4e32da8cd2efad, scope.project.id=f666a617f8d14eb18bbeb076032b82f2, user.id=854ab2dcec5a48b18f409b33cb61b5da) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:55.597889 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77000bb7-e76c-41c3-aeb4-d41ac6101ac5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:55.599348 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77000bb7-e76c-41c3-aeb4-d41ac6101ac5 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:55.607036 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 430/832] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 14:00:55 2022] GET /identity/v3/role_assignments?user.id=854ab2dcec5a48b18f409b33cb61b5da&scope.project.id=f666a617f8d14eb18bbeb076032b82f2&role.id=e14564f00daf48a28d4e32da8cd2efad => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:56.498093 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0d613-a371-4587-b467-92742fa26d05 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:56.498615 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0d613-a371-4587-b467-92742fa26d05 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:56.498615 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0d613-a371-4587-b467-92742fa26d05 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:56.500258 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 403/833] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:56.505558 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5f1fa-c14b-4e08-853a-1ab2259d5208 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:56.505975 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5f1fa-c14b-4e08-853a-1ab2259d5208 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:56.506151 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5f1fa-c14b-4e08-853a-1ab2259d5208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:56.525080 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-f1d5f1fa-c14b-4e08-853a-1ab2259d5208 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:56.526282 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f1d5f1fa-c14b-4e08-853a-1ab2259d5208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:56.599092 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 431/834] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:00:56.712213 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b9044-52a0-4b7b-8c3b-054500084fb3 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:56.712538 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b9044-52a0-4b7b-8c3b-054500084fb3 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:56.712760 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b9044-52a0-4b7b-8c3b-054500084fb3 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:56.714336 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 404/835] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:56.719513 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a063e5-67a4-4c01-acb0-f78776b6ca3a None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:56.719802 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a063e5-67a4-4c01-acb0-f78776b6ca3a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:56.720079 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a063e5-67a4-4c01-acb0-f78776b6ca3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:56.738768 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-23a063e5-67a4-4c01-acb0-f78776b6ca3a None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:56.739999 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-23a063e5-67a4-4c01-acb0-f78776b6ca3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:56.808375 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 432/836] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:00:56.867824 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b9044-52a0-4b7b-8c3b-054500084fb3 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:56.879737 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:56.978455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b9044-52a0-4b7b-8c3b-054500084fb3 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:57.013998 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9234ec-ab3a-409b-9b6d-943fd86411e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:57.014622 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9234ec-ab3a-409b-9b6d-943fd86411e9 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:57.014924 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9234ec-ab3a-409b-9b6d-943fd86411e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:57.015196 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9234ec-ab3a-409b-9b6d-943fd86411e9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:57.015775 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9234ec-ab3a-409b-9b6d-943fd86411e9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:57.017120 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9234ec-ab3a-409b-9b6d-943fd86411e9 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:57.026681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-6c9234ec-ab3a-409b-9b6d-943fd86411e9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 14:00:57.026681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:57.026681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 14:00:57.026681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 14:00:57.026681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 14:00:57.026681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 14:00:57.026681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 14:00:57.026681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:00:57.026681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 14:00:57.026681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:57.026681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 14:00:57.026681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:57.026681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:57.026681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:57.026681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:57.026681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:57.026681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:57.027697 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:57.027697 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:57.027697 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:57.027697 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:57.027697 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:57.027697 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:57.027697 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 14:00:57.027697 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 14:00:57.027697 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 14:00:57.027697 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 14:00:57.027697 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:57.027697 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:57.027697 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:57.027697 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:57.027697 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:57.027697 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:57.028537 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:57.028537 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:57.028537 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:57.028537 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:57.028537 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:57.028537 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:57.028537 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:57.028537 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:57.028537 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:57.028537 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:57.028537 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 14:00:57.028537 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:00:57.028537 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 14:00:57.028537 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:57.028537 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 405/837] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 12 14:00:56 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:57.035642 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a063e5-67a4-4c01-acb0-f78776b6ca3a None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:57.087904 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:57.124790 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9234ec-ab3a-409b-9b6d-943fd86411e9 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:57.140522 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a063e5-67a4-4c01-acb0-f78776b6ca3a None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:57.175058 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6a5c0-21d9-411d-9910-b8c54c73a1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:57.175669 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6a5c0-21d9-411d-9910-b8c54c73a1f9 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:57.175949 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6a5c0-21d9-411d-9910-b8c54c73a1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:57.176218 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6a5c0-21d9-411d-9910-b8c54c73a1f9 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:57.176850 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae6a5c0-21d9-411d-9910-b8c54c73a1f9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:57.177152 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae6a5c0-21d9-411d-9910-b8c54c73a1f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:57.178511 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae6a5c0-21d9-411d-9910-b8c54c73a1f9 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:57.183935 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 433/838] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 12 14:00:57 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:57.191795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6a5c0-21d9-411d-9910-b8c54c73a1f9 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:57.207479 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:57.238656 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9234ec-ab3a-409b-9b6d-943fd86411e9 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:57.275488 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ea089-cec7-4579-b4ed-3e8aadc7c2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:57.276120 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ea089-cec7-4579-b4ed-3e8aadc7c2bc None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:57.276394 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ea089-cec7-4579-b4ed-3e8aadc7c2bc None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:57.276663 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ea089-cec7-4579-b4ed-3e8aadc7c2bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:57.287357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566ea089-cec7-4579-b4ed-3e8aadc7c2bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:57.292324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-566ea089-cec7-4579-b4ed-3e8aadc7c2bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:00:57.292324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:57.292324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:57.292324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:57.292324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:57.292324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:57.292324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:57.292324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:57.292324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:57.292324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:57.292324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:57.292324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:57.292324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:57.292324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:57.292324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:57.292324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:57.293265 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:57.293265 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:57.293265 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:57.293265 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:57.293265 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:57.293265 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:57.293265 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:57.293265 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:57.293265 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:57.293265 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:57.293265 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:57.293265 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:57.293265 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:57.293265 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:57.293265 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:57.293265 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:57.294093 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:57.294093 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:57.294093 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:57.294093 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:57.294093 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:57.294093 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:57.294093 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:57.294093 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:57.294093 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:57.294093 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:57.294093 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:00:57.294093 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:57.294093 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 406/839] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 12 14:00:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:57.301532 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ea089-cec7-4579-b4ed-3e8aadc7c2bc None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:57.321911 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6a5c0-21d9-411d-9910-b8c54c73a1f9 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:57.338995 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:57.366316 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff22357f-eca5-416e-9744-77e54c8bcc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:57.367859 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22357f-eca5-416e-9744-77e54c8bcc5e None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:57.368470 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22357f-eca5-416e-9744-77e54c8bcc5e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:57.368958 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22357f-eca5-416e-9744-77e54c8bcc5e None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:57.369786 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff22357f-eca5-416e-9744-77e54c8bcc5e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:57.370282 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff22357f-eca5-416e-9744-77e54c8bcc5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:57.372664 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff22357f-eca5-416e-9744-77e54c8bcc5e None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:57.379393 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 434/840] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 12 14:00:57 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:57.425815 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ea089-cec7-4579-b4ed-3e8aadc7c2bc None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:57.461455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:57.469064 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a411d-febb-487b-90aa-747cc1708623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:57.470036 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a411d-febb-487b-90aa-747cc1708623 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:57.470519 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a411d-febb-487b-90aa-747cc1708623 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:57.471052 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a411d-febb-487b-90aa-747cc1708623 None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:57.471966 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674a411d-febb-487b-90aa-747cc1708623 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:57.472472 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674a411d-febb-487b-90aa-747cc1708623 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:57.474141 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674a411d-febb-487b-90aa-747cc1708623 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:57.485348 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 407/841] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 14:00:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:00:57.492332 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff22357f-eca5-416e-9744-77e54c8bcc5e None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:57.617404 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff22357f-eca5-416e-9744-77e54c8bcc5e None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:57.654358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c88578-df99-4ae4-9d0a-4c7e7bcf450b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:57.654702 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e619ac-1f03-4557-91b5-7a62a852fd47 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:57.655163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e619ac-1f03-4557-91b5-7a62a852fd47 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:57.655239 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c88578-df99-4ae4-9d0a-4c7e7bcf450b None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:57.655418 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e619ac-1f03-4557-91b5-7a62a852fd47 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:57.655496 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c88578-df99-4ae4-9d0a-4c7e7bcf450b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:57.655566 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c88578-df99-4ae4-9d0a-4c7e7bcf450b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:57.656772 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 408/842] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 14:00:57.661915 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ebed54-f3c1-4896-80c5-3049d7ab4ae2 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:57.662443 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ebed54-f3c1-4896-80c5-3049d7ab4ae2 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:57.662944 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ebed54-f3c1-4896-80c5-3049d7ab4ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:57.667935 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c88578-df99-4ae4-9d0a-4c7e7bcf450b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:57.669262 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c88578-df99-4ae4-9d0a-4c7e7bcf450b None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:57.683376 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-29ebed54-f3c1-4896-80c5-3049d7ab4ae2 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:57.684938 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-29ebed54-f3c1-4896-80c5-3049d7ab4ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:57.686070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-73c88578-df99-4ae4-9d0a-4c7e7bcf450b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:57.686070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:57.686070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:57.686070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:57.686070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:57.686070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:57.686070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:57.686070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:57.686070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:57.686070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:57.686070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:57.686070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:57.686070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:57.686070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:57.686070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:57.686070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:57.687221 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:57.687221 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:57.687221 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:57.687221 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:57.687221 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:57.687221 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:57.687221 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:57.687221 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:57.687221 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:57.687221 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:57.687221 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:57.687221 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:57.687221 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:57.687221 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:57.687221 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:57.687221 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:57.687221 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:57.688325 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:57.688325 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:57.688325 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:57.688325 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 435/843] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:57.693098 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c88578-df99-4ae4-9d0a-4c7e7bcf450b None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:57.724440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:57.790899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 409/844] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:57 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:00:57.812312 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c88578-df99-4ae4-9d0a-4c7e7bcf450b None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:57.860289 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c29c3-f9b0-487a-a25f-b53c307a4156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:57.861249 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c29c3-f9b0-487a-a25f-b53c307a4156 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:57.861732 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c29c3-f9b0-487a-a25f-b53c307a4156 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:57.862148 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c29c3-f9b0-487a-a25f-b53c307a4156 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:57.862970 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526c29c3-f9b0-487a-a25f-b53c307a4156 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:57.863418 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526c29c3-f9b0-487a-a25f-b53c307a4156 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:57.864935 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526c29c3-f9b0-487a-a25f-b53c307a4156 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:57.878000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 436/845] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:00:57.883788 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ebed54-f3c1-4896-80c5-3049d7ab4ae2 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:57.889095 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df1cd4-ac68-49fd-9413-6e239229a03f None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:57.889375 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df1cd4-ac68-49fd-9413-6e239229a03f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:57.889646 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df1cd4-ac68-49fd-9413-6e239229a03f None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:57.891231 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 437/846] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 14:00:57.896143 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde8117-18bc-4ab4-b284-6bf03b53af8d None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:57.896433 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde8117-18bc-4ab4-b284-6bf03b53af8d None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:57.896695 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde8117-18bc-4ab4-b284-6bf03b53af8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:57.916602 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-acde8117-18bc-4ab4-b284-6bf03b53af8d None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:57.916799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-acde8117-18bc-4ab4-b284-6bf03b53af8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:57.993708 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 438/847] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:57 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 14:00:58.005867 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ebed54-f3c1-4896-80c5-3049d7ab4ae2 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:58.030686 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:58.042480 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2ebe3a-fbb8-4549-8ee4-54d1e69f1833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:58.043133 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ebe3a-fbb8-4549-8ee4-54d1e69f1833 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:58.043406 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ebe3a-fbb8-4549-8ee4-54d1e69f1833 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:58.043672 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ebe3a-fbb8-4549-8ee4-54d1e69f1833 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:58.050775 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2ebe3a-fbb8-4549-8ee4-54d1e69f1833 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:58.052204 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2ebe3a-fbb8-4549-8ee4-54d1e69f1833 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:58.064341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-bc2ebe3a-fbb8-4549-8ee4-54d1e69f1833 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 12 14:00:58.064341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:58.064341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:58.064341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:58.064341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:58.064341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:58.064341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:58.064341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:58.064341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:58.064341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:58.064341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:58.064341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:58.064341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:00:58.064341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:00:58.064341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:00:58.064341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:00:58.065269 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:58.065269 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:58.065269 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:00:58.065269 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:58.065269 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:00:58.065269 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:58.065269 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:58.065269 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:58.065269 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:58.065269 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:58.065269 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:58.065269 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:58.065269 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:58.065269 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:58.065269 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:58.065269 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:58.065269 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:58.065269 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:58.066233 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:58.066233 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:58.066233 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:00:58.066233 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:00:58.066233 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:00:58.066233 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:00:58.066233 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:00:58.066233 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:00:58.066233 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 12 14:00:58.066233 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:58.066233 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 410/848] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 12 14:00:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:58.073777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acde8117-18bc-4ab4-b284-6bf03b53af8d None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:58.090908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2ebe3a-fbb8-4549-8ee4-54d1e69f1833 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:58.103109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:58.193154 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acde8117-18bc-4ab4-b284-6bf03b53af8d None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:58.220823 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2ebe3a-fbb8-4549-8ee4-54d1e69f1833 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:58.233175 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6edde5-555b-4c22-bc3b-beb7eff9ede9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:58.234056 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6edde5-555b-4c22-bc3b-beb7eff9ede9 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:58.234900 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6edde5-555b-4c22-bc3b-beb7eff9ede9 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:58.235648 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6edde5-555b-4c22-bc3b-beb7eff9ede9 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:58.236574 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6edde5-555b-4c22-bc3b-beb7eff9ede9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:58.237070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6edde5-555b-4c22-bc3b-beb7eff9ede9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:58.238846 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6edde5-555b-4c22-bc3b-beb7eff9ede9 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:58.261898 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 439/849] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Sep 12 14:00:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:58.266796 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134fda12-5f24-4469-a48d-853e9345ff9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:58.267077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6edde5-555b-4c22-bc3b-beb7eff9ede9 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:58.267837 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134fda12-5f24-4469-a48d-853e9345ff9f None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:58.268247 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134fda12-5f24-4469-a48d-853e9345ff9f None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:58.268713 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134fda12-5f24-4469-a48d-853e9345ff9f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:58.282802 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134fda12-5f24-4469-a48d-853e9345ff9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:00:58.291885 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-134fda12-5f24-4469-a48d-853e9345ff9f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:00:58.291885 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:58.291885 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:58.291885 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:58.291885 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:58.291885 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:58.291885 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:58.291885 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:58.291885 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:58.291885 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:58.291885 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:58.291885 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:58.291885 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:00:58.291885 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:00:58.291885 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:00:58.291885 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:00:58.295467 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:00:58.295467 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:00:58.295467 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:00:58.295467 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:00:58.295467 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:58.295467 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:58.295467 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:58.295467 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:58.295467 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:58.295467 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:58.295467 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:58.295467 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:58.295467 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:58.295467 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:58.295467 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:58.295467 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:58.296280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:58.296280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:58.296280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:58.296280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:58.296280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:00:58.296280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:00:58.296280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:00:58.296280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:00:58.296280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:00:58.296280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:00:58.296280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:00:58.296280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:58.296280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 411/850] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 12 14:00:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:00:58.296280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134fda12-5f24-4469-a48d-853e9345ff9f None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:58.343232 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:58.361791 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:58.385602 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6edde5-555b-4c22-bc3b-beb7eff9ede9 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:58.403049 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134fda12-5f24-4469-a48d-853e9345ff9f None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:58.425498 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf3312-7655-4ddc-b07e-8950d163f586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:58.426104 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf3312-7655-4ddc-b07e-8950d163f586 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:58.426380 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf3312-7655-4ddc-b07e-8950d163f586 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:58.426649 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf3312-7655-4ddc-b07e-8950d163f586 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:58.441394 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481b70d-d611-4949-bfb5-fca46b9edb59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:58.442017 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481b70d-d611-4949-bfb5-fca46b9edb59 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:58.442288 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481b70d-d611-4949-bfb5-fca46b9edb59 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:58.442556 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481b70d-d611-4949-bfb5-fca46b9edb59 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:58.443207 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3481b70d-d611-4949-bfb5-fca46b9edb59 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:58.444508 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3481b70d-d611-4949-bfb5-fca46b9edb59 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:58.453626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-3481b70d-d611-4949-bfb5-fca46b9edb59 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 14:00:58.453626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:58.453626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 14:00:58.453626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 14:00:58.453626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 14:00:58.453626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 14:00:58.453626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 14:00:58.453626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:00:58.453626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 14:00:58.453626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:58.453626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 14:00:58.453626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:58.453626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:58.453626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:58.453626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:58.453626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:58.454591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:58.454591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:58.454591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:58.454591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:58.454591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:58.454591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:58.454591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:58.454591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 14:00:58.454591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 14:00:58.454591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 14:00:58.454591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 14:00:58.454591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:58.454591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:58.454591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:58.454591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:58.454591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:58.454591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:58.455845 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:58.455845 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:58.455845 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:58.455845 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:58.455845 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:58.455845 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:58.455845 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:58.455845 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:58.455845 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:58.455845 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:58.455845 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 14:00:58.455845 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:00:58.455845 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 14:00:58.455845 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:58.455845 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 412/851] 10.222.0.42 () {56 vars in 1132 bytes} [Mon Sep 12 14:00:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:58.456709 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481b70d-d611-4949-bfb5-fca46b9edb59 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:58.504887 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf3312-7655-4ddc-b07e-8950d163f586 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:58.506310 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf3312-7655-4ddc-b07e-8950d163f586 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:58.560208 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481b70d-d611-4949-bfb5-fca46b9edb59 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:58.570043 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:58.586403 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:58.598342 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab43ca9-26bf-45f6-8243-2d648658ffd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:58.598968 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab43ca9-26bf-45f6-8243-2d648658ffd7 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:58.599240 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab43ca9-26bf-45f6-8243-2d648658ffd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:58.599500 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab43ca9-26bf-45f6-8243-2d648658ffd7 None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:58.600159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab43ca9-26bf-45f6-8243-2d648658ffd7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:58.600450 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab43ca9-26bf-45f6-8243-2d648658ffd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:58.602391 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab43ca9-26bf-45f6-8243-2d648658ffd7 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:58.609930 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 440/852] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:00:58 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 14:00:58.610707 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 413/853] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 12 14:00:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:58.615423 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf3312-7655-4ddc-b07e-8950d163f586 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:58.615554 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab43ca9-26bf-45f6-8243-2d648658ffd7 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:58.717979 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab43ca9-26bf-45f6-8243-2d648658ffd7 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:58.728047 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf3312-7655-4ddc-b07e-8950d163f586 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:58.753243 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216492a-5045-447a-b694-e8190584c685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:58.753962 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216492a-5045-447a-b694-e8190584c685 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:58.754247 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216492a-5045-447a-b694-e8190584c685 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:58.754521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216492a-5045-447a-b694-e8190584c685 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:58.764215 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b1dddc-4043-4476-bdcf-7262e24ddfd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:58.764818 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1dddc-4043-4476-bdcf-7262e24ddfd9 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:58.765096 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1dddc-4043-4476-bdcf-7262e24ddfd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:58.765365 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1dddc-4043-4476-bdcf-7262e24ddfd9 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:58.765464 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e216492a-5045-447a-b694-e8190584c685 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:58.765986 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b1dddc-4043-4476-bdcf-7262e24ddfd9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:58.766286 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b1dddc-4043-4476-bdcf-7262e24ddfd9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:58.766794 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e216492a-5045-447a-b694-e8190584c685 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:58.767643 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b1dddc-4043-4476-bdcf-7262e24ddfd9 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:58.777713 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 441/854] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Sep 12 14:00:58 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:58.778572 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b1dddc-4043-4476-bdcf-7262e24ddfd9 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:58.782503 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-e216492a-5045-447a-b694-e8190584c685 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:58.782503 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:58.782503 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:58.782503 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:58.782503 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:58.782503 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:58.782503 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:58.782503 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:58.782503 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:58.782503 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:58.782503 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:58.782503 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:58.782503 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:58.782503 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:58.782503 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:58.782503 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:58.783546 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:58.783546 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:58.783546 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:58.783546 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:58.783546 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:58.783546 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:58.783546 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:58.783546 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:58.783546 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:58.783546 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:58.783546 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:58.783546 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:58.783546 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:58.783546 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:58.783546 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:58.783546 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:58.783546 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:58.784424 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:58.784424 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:58.784424 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:58.784424 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 414/855] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:58.784424 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216492a-5045-447a-b694-e8190584c685 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:58.807882 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:58.839345 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:58.885985 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216492a-5045-447a-b694-e8190584c685 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:58.899290 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b1dddc-4043-4476-bdcf-7262e24ddfd9 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:58.921964 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246dd3c3-068d-4061-9067-e1b0a43864a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:58.922560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246dd3c3-068d-4061-9067-e1b0a43864a4 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:58.922851 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246dd3c3-068d-4061-9067-e1b0a43864a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:58.923128 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246dd3c3-068d-4061-9067-e1b0a43864a4 None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:58.923730 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246dd3c3-068d-4061-9067-e1b0a43864a4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:58.924031 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246dd3c3-068d-4061-9067-e1b0a43864a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:58.925372 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246dd3c3-068d-4061-9067-e1b0a43864a4 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:58.938871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 415/856] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Sep 12 14:00:58 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:58.938871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246dd3c3-068d-4061-9067-e1b0a43864a4 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:58.942585 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b554732c-cab5-44a2-9257-2d9138609ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:58.943360 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554732c-cab5-44a2-9257-2d9138609ad1 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:58.943705 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554732c-cab5-44a2-9257-2d9138609ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:58.944045 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554732c-cab5-44a2-9257-2d9138609ad1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:58.956640 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b554732c-cab5-44a2-9257-2d9138609ad1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:58.958874 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b554732c-cab5-44a2-9257-2d9138609ad1 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:58.977555 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-b554732c-cab5-44a2-9257-2d9138609ad1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:58.977555 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:58.977555 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:58.977555 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:58.977555 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:58.977555 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:58.977555 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:58.977555 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:58.977555 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:58.977555 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:58.977555 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:58.977555 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:58.977555 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:00:58.977555 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:00:58.977555 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:00:58.977555 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:00:58.978560 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:58.978560 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:58.978560 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:58.978560 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:58.978560 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:58.978560 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:58.978560 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:58.978560 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:58.978560 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:58.978560 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:58.978560 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:58.978560 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:58.978560 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:58.978560 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:58.978560 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:58.978560 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:58.978560 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:00:58.979514 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:00:58.979514 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:00:58.979514 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:58.979514 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 442/857] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:58.987924 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b554732c-cab5-44a2-9257-2d9138609ad1 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:59.043282 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:59.061403 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246dd3c3-068d-4061-9067-e1b0a43864a4 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:59.099000 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6176a-a7b6-4b19-a99e-c8b2e2e1b9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:59.099805 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6176a-a7b6-4b19-a99e-c8b2e2e1b9e2 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:59.100184 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6176a-a7b6-4b19-a99e-c8b2e2e1b9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:59.100598 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6176a-a7b6-4b19-a99e-c8b2e2e1b9e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:59.101868 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb6176a-a7b6-4b19-a99e-c8b2e2e1b9e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:59.102259 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb6176a-a7b6-4b19-a99e-c8b2e2e1b9e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:59.103797 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb6176a-a7b6-4b19-a99e-c8b2e2e1b9e2 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:59.118163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 416/858] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:59.121230 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:59.123978 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6176a-a7b6-4b19-a99e-c8b2e2e1b9e2 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:59.130713 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b554732c-cab5-44a2-9257-2d9138609ad1 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:59.170109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0eeda1-822e-45ae-9047-39c56bbf83b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:59.170995 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0eeda1-822e-45ae-9047-39c56bbf83b2 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:59.171490 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0eeda1-822e-45ae-9047-39c56bbf83b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:59.172261 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0eeda1-822e-45ae-9047-39c56bbf83b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:59.173051 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0eeda1-822e-45ae-9047-39c56bbf83b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:59.173804 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0eeda1-822e-45ae-9047-39c56bbf83b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:59.175620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0eeda1-822e-45ae-9047-39c56bbf83b2 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:59.190544 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 443/859] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:59.201679 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0eeda1-822e-45ae-9047-39c56bbf83b2 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:59.247327 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6176a-a7b6-4b19-a99e-c8b2e2e1b9e2 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:59.294529 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2252039-6fd7-4c4b-be16-4bd03c24b29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:59.295351 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2252039-6fd7-4c4b-be16-4bd03c24b29a None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:59.295728 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2252039-6fd7-4c4b-be16-4bd03c24b29a None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:59.296126 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2252039-6fd7-4c4b-be16-4bd03c24b29a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:59.306365 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2252039-6fd7-4c4b-be16-4bd03c24b29a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:59.308725 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2252039-6fd7-4c4b-be16-4bd03c24b29a None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:59.309910 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0eeda1-822e-45ae-9047-39c56bbf83b2 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:59.324252 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-e2252039-6fd7-4c4b-be16-4bd03c24b29a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:59.324252 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:59.324252 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:59.324252 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:59.324252 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:59.324252 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:59.324252 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:59.324252 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:59.324252 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:59.324252 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:59.324252 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:59.324252 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:59.324252 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:00:59.324252 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:00:59.324252 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:00:59.325176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:00:59.325176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:59.325176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:59.325176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:59.325176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:59.325176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:59.325176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:59.325176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:59.325176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:59.325176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:59.325176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:59.325176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:59.325176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:59.325176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:59.325176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:59.325176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:59.325176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:59.326065 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:00:59.326065 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:00:59.326065 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:00:59.326065 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:00:59.326065 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:00:59.326065 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:00:59.326065 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:00:59.326065 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:00:59.330918 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 417/860] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:00:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:00:59.334212 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2252039-6fd7-4c4b-be16-4bd03c24b29a None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:59.346022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:59.357261 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd3f68-ade6-48ba-b523-4f7e61d2d5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:59.357936 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd3f68-ade6-48ba-b523-4f7e61d2d5c8 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:59.358238 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd3f68-ade6-48ba-b523-4f7e61d2d5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:59.358505 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd3f68-ade6-48ba-b523-4f7e61d2d5c8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:59.367462 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd3f68-ade6-48ba-b523-4f7e61d2d5c8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:59.368881 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd3f68-ade6-48ba-b523-4f7e61d2d5c8 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:59.383574 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-abcd3f68-ade6-48ba-b523-4f7e61d2d5c8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 14:00:59.383574 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:00:59.383574 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:00:59.383574 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:00:59.383574 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:00:59.383574 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:00:59.383574 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:00:59.383574 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:00:59.383574 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:00:59.383574 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:00:59.383574 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:00:59.383574 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:00:59.383574 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:00:59.383574 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:00:59.383574 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:00:59.383574 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:00:59.385765 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:00:59.385765 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:00:59.385765 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:00:59.385765 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:59.385765 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:00:59.385765 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:00:59.385765 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:00:59.385765 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:00:59.385765 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:00:59.385765 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:00:59.385765 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:00:59.385765 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:00:59.385765 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:00:59.385765 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:00:59.385765 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:00:59.385765 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:00:59.385765 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:00:59.385765 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:00:59.387957 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:00:59.387957 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:00:59.387957 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:00:59.387957 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:00:59.387957 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:00:59.387957 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:00:59.387957 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:00:59.387957 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:00:59.387957 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 14:00:59.387957 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:00:59.387957 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 444/861] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Sep 12 14:00:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:00:59.396072 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd3f68-ade6-48ba-b523-4f7e61d2d5c8 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:59.415915 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:59.466119 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2252039-6fd7-4c4b-be16-4bd03c24b29a None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:59.512245 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e4d51-a8ea-42e9-9485-69dae7975cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:59.513522 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e4d51-a8ea-42e9-9485-69dae7975cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:59.513812 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e4d51-a8ea-42e9-9485-69dae7975cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:59.514085 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e4d51-a8ea-42e9-9485-69dae7975cf1 None admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:59.514891 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29e4d51-a8ea-42e9-9485-69dae7975cf1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:59.515207 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29e4d51-a8ea-42e9-9485-69dae7975cf1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:59.515588 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd3f68-ade6-48ba-b523-4f7e61d2d5c8 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:59.516732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29e4d51-a8ea-42e9-9485-69dae7975cf1 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:59.541146 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 418/862] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 12 14:00:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:59.546283 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e4d51-a8ea-42e9-9485-69dae7975cf1 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:59.562231 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892a3a7-b2eb-483d-abe9-bdf4af79fc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:59.562923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892a3a7-b2eb-483d-abe9-bdf4af79fc15 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:59.563250 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892a3a7-b2eb-483d-abe9-bdf4af79fc15 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:59.563521 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892a3a7-b2eb-483d-abe9-bdf4af79fc15 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:59.564295 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1892a3a7-b2eb-483d-abe9-bdf4af79fc15 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:59.564601 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1892a3a7-b2eb-483d-abe9-bdf4af79fc15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:59.566679 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1892a3a7-b2eb-483d-abe9-bdf4af79fc15 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:59.587798 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 445/863] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Sep 12 14:00:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:59.593863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892a3a7-b2eb-483d-abe9-bdf4af79fc15 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:00:59.628373 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:59.667371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e4d51-a8ea-42e9-9485-69dae7975cf1 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:59.684686 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:59.701776 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a676d-e3c1-4e86-b492-9f7b02a7d44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:59.702373 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a676d-e3c1-4e86-b492-9f7b02a7d44b None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:59.702648 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a676d-e3c1-4e86-b492-9f7b02a7d44b None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:59.702955 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a676d-e3c1-4e86-b492-9f7b02a7d44b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:59.703668 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a676d-e3c1-4e86-b492-9f7b02a7d44b None admin] RBAC: Adding query filter params (role.id=e14564f00daf48a28d4e32da8cd2efad, scope.project.id=f666a617f8d14eb18bbeb076032b82f2, user.id=25a87ea9c3c5409b97a35c4857ae3ca6) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:00:59.703968 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a676d-e3c1-4e86-b492-9f7b02a7d44b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:59.705370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a676d-e3c1-4e86-b492-9f7b02a7d44b None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:59.722871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 419/864] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 14:00:59 2022] GET /identity/v3/role_assignments?user.id=25a87ea9c3c5409b97a35c4857ae3ca6&scope.project.id=f666a617f8d14eb18bbeb076032b82f2&role.id=e14564f00daf48a28d4e32da8cd2efad => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:00:59.722871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e03ab43-808c-4564-9187-e8f1ac9810d9 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:59.722871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e03ab43-808c-4564-9187-e8f1ac9810d9 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:59.722871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e03ab43-808c-4564-9187-e8f1ac9810d9 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:59.722871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 420/865] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:00:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:00:59.722871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477dfcf7-18d8-4f8e-9a10-2ac480c64a0b None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:59.723826 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477dfcf7-18d8-4f8e-9a10-2ac480c64a0b None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:59.723826 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477dfcf7-18d8-4f8e-9a10-2ac480c64a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:59.726920 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892a3a7-b2eb-483d-abe9-bdf4af79fc15 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:00:59.741630 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-477dfcf7-18d8-4f8e-9a10-2ac480c64a0b None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:00:59.743008 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-477dfcf7-18d8-4f8e-9a10-2ac480c64a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:00:59.765860 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1807f8-f434-4061-bac5-d41aa675bd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:00:59.766395 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1807f8-f434-4061-bac5-d41aa675bd30 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:00:59.766653 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1807f8-f434-4061-bac5-d41aa675bd30 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:00:59.767054 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1807f8-f434-4061-bac5-d41aa675bd30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:00:59.843607 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1807f8-f434-4061-bac5-d41aa675bd30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:00:59.843853 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1807f8-f434-4061-bac5-d41aa675bd30 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:00:59.848250 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 421/866] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:00:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 14:00:59.927761 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:00:59.962686 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 446/867] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:00:59 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 14:00:59.968091 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477dfcf7-18d8-4f8e-9a10-2ac480c64a0b None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:00.031066 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:00.076381 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477dfcf7-18d8-4f8e-9a10-2ac480c64a0b None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:00.115746 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1807f8-f434-4061-bac5-d41aa675bd30 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:00.124528 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b9a0c9-1c50-4945-9061-a538e54778fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:00.125444 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9a0c9-1c50-4945-9061-a538e54778fa None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:00.125922 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9a0c9-1c50-4945-9061-a538e54778fa None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:00.126531 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9a0c9-1c50-4945-9061-a538e54778fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:00.138533 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b9a0c9-1c50-4945-9061-a538e54778fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:00.140268 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b9a0c9-1c50-4945-9061-a538e54778fa None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:00.156887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-38b9a0c9-1c50-4945-9061-a538e54778fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:01:00.156887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:00.156887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:00.156887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:00.156887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:00.156887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:00.156887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:00.156887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:00.156887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:00.156887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:00.156887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:00.156887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:00.156887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:01:00.156887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:01:00.156887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:01:00.156887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:01:00.157880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:00.157880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:00.157880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:00.157880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:00.157880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:00.157880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:00.157880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:00.157880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:00.157880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:00.157880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:00.157880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:00.157880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:00.157880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:00.157880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:00.157880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:00.157880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:00.157880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:01:00.158770 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:01:00.158770 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:01:00.158770 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:00.161070 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 422/868] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:00:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:00.167299 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b9a0c9-1c50-4945-9061-a538e54778fa None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:00.222808 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1807f8-f434-4061-bac5-d41aa675bd30 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:00.259151 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b5a6e-1182-4c10-a0ea-aa3cb3690a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:00.259759 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b5a6e-1182-4c10-a0ea-aa3cb3690a70 None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:00.260040 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b5a6e-1182-4c10-a0ea-aa3cb3690a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:00.260311 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b5a6e-1182-4c10-a0ea-aa3cb3690a70 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:00.260975 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b5a6e-1182-4c10-a0ea-aa3cb3690a70 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:00.262444 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b5a6e-1182-4c10-a0ea-aa3cb3690a70 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:00.275418 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b9a0c9-1c50-4945-9061-a538e54778fa None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:00.281934 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 447/869] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Sep 12 14:01:00 2022] POST /identity/v3/services => generated 251 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:01:00.295651 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:00.329805 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc9c726-2287-4143-aaa0-3265e23f8c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:00.330728 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9c726-2287-4143-aaa0-3265e23f8c95 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:00.331185 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9c726-2287-4143-aaa0-3265e23f8c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:00.331643 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9c726-2287-4143-aaa0-3265e23f8c95 None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:00.332483 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc9c726-2287-4143-aaa0-3265e23f8c95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:00.332979 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc9c726-2287-4143-aaa0-3265e23f8c95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:00.334577 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc9c726-2287-4143-aaa0-3265e23f8c95 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:00.349880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 423/870] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:01:00.357095 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b5a6e-1182-4c10-a0ea-aa3cb3690a70 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:00.394623 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:00.475014 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b5a6e-1182-4c10-a0ea-aa3cb3690a70 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:00.512714 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419ac2f8-fbdf-4dd0-a6bd-0227d7233075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:00.513348 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ac2f8-fbdf-4dd0-a6bd-0227d7233075 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:00.513630 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ac2f8-fbdf-4dd0-a6bd-0227d7233075 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:00.513901 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ac2f8-fbdf-4dd0-a6bd-0227d7233075 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:00.520792 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419ac2f8-fbdf-4dd0-a6bd-0227d7233075 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:00.522228 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419ac2f8-fbdf-4dd0-a6bd-0227d7233075 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:00.534826 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-419ac2f8-fbdf-4dd0-a6bd-0227d7233075 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:01:00.534826 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:00.534826 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:00.534826 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:00.534826 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:00.534826 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:00.534826 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:00.534826 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:00.534826 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:00.534826 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:00.534826 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:00.534826 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:00.534826 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:01:00.534826 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:01:00.534826 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:01:00.535844 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:01:00.535844 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:00.535844 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:00.535844 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:00.535844 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:00.535844 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:00.535844 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:00.535844 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:00.535844 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:00.535844 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:00.535844 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:00.535844 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:00.535844 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:00.535844 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:00.535844 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:00.535844 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:00.535844 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:00.536839 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:01:00.536839 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:01:00.536839 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:01:00.536839 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:01:00.536839 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:01:00.536839 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:01:00.536839 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:01:00.536839 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:00.536839 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 448/871] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:01:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:00.543577 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc9c726-2287-4143-aaa0-3265e23f8c95 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:00.648973 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc9c726-2287-4143-aaa0-3265e23f8c95 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:00.685239 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87d4ba-fb8f-445c-945b-7623d3a423e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:00.686214 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87d4ba-fb8f-445c-945b-7623d3a423e4 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:00.686706 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87d4ba-fb8f-445c-945b-7623d3a423e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:00.687232 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87d4ba-fb8f-445c-945b-7623d3a423e4 None admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:00.688132 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea87d4ba-fb8f-445c-945b-7623d3a423e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:00.688646 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea87d4ba-fb8f-445c-945b-7623d3a423e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:00.690294 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea87d4ba-fb8f-445c-945b-7623d3a423e4 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:00.703350 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 424/872] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 12 14:01:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:00.709461 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419ac2f8-fbdf-4dd0-a6bd-0227d7233075 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:00.807038 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:00.816401 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419ac2f8-fbdf-4dd0-a6bd-0227d7233075 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:00.852585 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a4fdac-cf27-480a-9838-d02023f7554f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:00.853184 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4fdac-cf27-480a-9838-d02023f7554f None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:00.853456 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4fdac-cf27-480a-9838-d02023f7554f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:00.853720 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4fdac-cf27-480a-9838-d02023f7554f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:00.854439 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a4fdac-cf27-480a-9838-d02023f7554f None admin] RBAC: Adding query filter params (role.id=1c413f4e78ef45b7a821bb884f9cea07, scope.project.id=f666a617f8d14eb18bbeb076032b82f2, user.id=1405e758a3af4e8ba659f498cd28ace1) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:00.854741 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a4fdac-cf27-480a-9838-d02023f7554f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:00.856365 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a4fdac-cf27-480a-9838-d02023f7554f None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:00.862619 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 449/873] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 14:01:00 2022] GET /identity/v3/role_assignments?user.id=1405e758a3af4e8ba659f498cd28ace1&scope.project.id=f666a617f8d14eb18bbeb076032b82f2&role.id=1c413f4e78ef45b7a821bb884f9cea07 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:01.201313 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542df71-ef9c-49ec-9986-35cfe51cdffd None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:01.201818 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542df71-ef9c-49ec-9986-35cfe51cdffd None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:01.201818 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542df71-ef9c-49ec-9986-35cfe51cdffd None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:01.203333 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 425/874] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:01:01.208746 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15434b46-3722-4a0b-a32e-22666676f876 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:01.209032 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15434b46-3722-4a0b-a32e-22666676f876 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:01.209306 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15434b46-3722-4a0b-a32e-22666676f876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:01.227786 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-15434b46-3722-4a0b-a32e-22666676f876 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:01.228963 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-15434b46-3722-4a0b-a32e-22666676f876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:01.297363 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 450/875] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:01 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:01:01.563013 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2542df71-ef9c-49ec-9986-35cfe51cdffd None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:01.597531 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:01.677025 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2542df71-ef9c-49ec-9986-35cfe51cdffd None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:01.715174 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed8dfc-9605-4ed9-ba51-059f2131203a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:01.715786 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed8dfc-9605-4ed9-ba51-059f2131203a None admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:01.716065 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed8dfc-9605-4ed9-ba51-059f2131203a None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:01.716335 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed8dfc-9605-4ed9-ba51-059f2131203a None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:01.716999 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ed8dfc-9605-4ed9-ba51-059f2131203a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:01.718354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ed8dfc-9605-4ed9-ba51-059f2131203a None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:01.736887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 426/876] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Sep 12 14:01:01 2022] POST /identity/v3/services => generated 269 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:01:02.123627 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae11038-e3e0-43e5-b8f1-558c9ab8b22b None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:02.124144 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae11038-e3e0-43e5-b8f1-558c9ab8b22b None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:02.124604 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae11038-e3e0-43e5-b8f1-558c9ab8b22b None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:02.126237 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 451/877] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:01:02.133289 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffd12d-922d-4a20-8bfe-4cd251532537 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:02.133574 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffd12d-922d-4a20-8bfe-4cd251532537 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:02.133846 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffd12d-922d-4a20-8bfe-4cd251532537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:02.152696 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-39ffd12d-922d-4a20-8bfe-4cd251532537 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:02.153887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-39ffd12d-922d-4a20-8bfe-4cd251532537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:02.230630 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 427/878] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:02 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:01:02.502148 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae11038-e3e0-43e5-b8f1-558c9ab8b22b None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:02.597346 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:02.613713 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae11038-e3e0-43e5-b8f1-558c9ab8b22b None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:02.617293 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4060942-2cab-4201-af9f-f267ab601917 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:02.617575 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4060942-2cab-4201-af9f-f267ab601917 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:02.617846 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4060942-2cab-4201-af9f-f267ab601917 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:02.619439 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 428/879] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:01:02.626485 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4b167-f01c-485b-97a5-9b675decaa6f None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:02.626995 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4b167-f01c-485b-97a5-9b675decaa6f None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:02.627605 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4b167-f01c-485b-97a5-9b675decaa6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:02.648799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc676584-bc46-4ccd-bd01-dbdbb2b2f567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:02.649429 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc676584-bc46-4ccd-bd01-dbdbb2b2f567 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:02.649703 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc676584-bc46-4ccd-bd01-dbdbb2b2f567 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:02.649971 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc676584-bc46-4ccd-bd01-dbdbb2b2f567 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:02.650551 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc676584-bc46-4ccd-bd01-dbdbb2b2f567 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:02.651930 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc676584-bc46-4ccd-bd01-dbdbb2b2f567 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:02.652519 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-31d4b167-f01c-485b-97a5-9b675decaa6f None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:02.654199 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-31d4b167-f01c-485b-97a5-9b675decaa6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:02.661065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-bc676584-bc46-4ccd-bd01-dbdbb2b2f567 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 14:01:02.661065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:02.661065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 14:01:02.661065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 14:01:02.661065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 14:01:02.661065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 14:01:02.661065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 14:01:02.661065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:01:02.661065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 14:01:02.661065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:02.661065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 14:01:02.661065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:02.661065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:02.661065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:02.661065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:02.661065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:02.661065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:02.662097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:02.662097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:02.662097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:02.662097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:02.662097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:02.662097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:02.662097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 14:01:02.662097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 14:01:02.662097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 14:01:02.662097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 14:01:02.662097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:02.662097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:02.662097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:02.662097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:02.662097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:02.662097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:02.662973 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:02.662973 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:02.662973 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:02.662973 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:02.662973 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:02.662973 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:02.662973 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:02.662973 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:02.662973 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:02.662973 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:02.662973 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 14:01:02.662973 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:01:02.662973 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 14:01:02.662973 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:02.662973 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 452/880] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Sep 12 14:01:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:02.671000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc676584-bc46-4ccd-bd01-dbdbb2b2f567 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:02.671920 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:02.746533 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 429/881] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:02 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:01:02.796594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc676584-bc46-4ccd-bd01-dbdbb2b2f567 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:02.831571 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8498f6-888d-4159-9cb9-82b3c73e8df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:02.832174 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8498f6-888d-4159-9cb9-82b3c73e8df6 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:02.832447 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8498f6-888d-4159-9cb9-82b3c73e8df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:02.832713 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8498f6-888d-4159-9cb9-82b3c73e8df6 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:02.833337 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8498f6-888d-4159-9cb9-82b3c73e8df6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:02.833630 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8498f6-888d-4159-9cb9-82b3c73e8df6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:02.835008 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8498f6-888d-4159-9cb9-82b3c73e8df6 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:02.840754 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 453/882] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Sep 12 14:01:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:01:02.846454 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4b167-f01c-485b-97a5-9b675decaa6f None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:02.951552 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4b167-f01c-485b-97a5-9b675decaa6f None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:02.988187 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973fbdf7-0b36-4515-8731-4043b665eb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:02.989132 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973fbdf7-0b36-4515-8731-4043b665eb34 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:02.989634 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973fbdf7-0b36-4515-8731-4043b665eb34 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:02.990112 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973fbdf7-0b36-4515-8731-4043b665eb34 None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:02.990970 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973fbdf7-0b36-4515-8731-4043b665eb34 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:02.991535 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973fbdf7-0b36-4515-8731-4043b665eb34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:02.993107 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973fbdf7-0b36-4515-8731-4043b665eb34 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:02.998397 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 430/883] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Sep 12 14:01:02 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:03.023369 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8498f6-888d-4159-9cb9-82b3c73e8df6 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:03.044976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:03.136604 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9cc001-d83d-4df9-90d0-3783faa927c1 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:03.137232 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9cc001-d83d-4df9-90d0-3783faa927c1 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:03.137704 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9cc001-d83d-4df9-90d0-3783faa927c1 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:03.139595 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 431/884] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 14:01:03.144803 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8498f6-888d-4159-9cb9-82b3c73e8df6 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:03.146051 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6393691c-dbd3-41f7-9208-3ce007280b83 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:03.146564 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6393691c-dbd3-41f7-9208-3ce007280b83 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:03.147070 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6393691c-dbd3-41f7-9208-3ce007280b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:03.166112 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-6393691c-dbd3-41f7-9208-3ce007280b83 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:03.167767 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-6393691c-dbd3-41f7-9208-3ce007280b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:03.170258 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:03.182224 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78cf4c5-3e28-4b9d-8cc2-21908e350c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:03.182875 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78cf4c5-3e28-4b9d-8cc2-21908e350c9b None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:03.183155 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78cf4c5-3e28-4b9d-8cc2-21908e350c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:03.183421 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78cf4c5-3e28-4b9d-8cc2-21908e350c9b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:03.184062 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78cf4c5-3e28-4b9d-8cc2-21908e350c9b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:03.185408 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78cf4c5-3e28-4b9d-8cc2-21908e350c9b None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:03.194489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-e78cf4c5-3e28-4b9d-8cc2-21908e350c9b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 14:01:03.194489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:03.194489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 14:01:03.194489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 14:01:03.194489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 14:01:03.194489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 14:01:03.194489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 14:01:03.194489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:01:03.194489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 14:01:03.194489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:03.194489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 14:01:03.194489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:03.194489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:03.194489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:03.194489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:03.194489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:03.194489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:03.196124 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:03.196124 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:03.196124 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:03.196124 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:03.196124 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:03.196124 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:03.196124 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 14:01:03.196124 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 14:01:03.196124 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 14:01:03.196124 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 14:01:03.196124 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:03.196124 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:03.196124 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:03.196124 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:03.196124 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:03.196124 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:03.197223 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:03.197223 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:03.197223 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:03.197223 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:03.197223 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:03.197223 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:03.197223 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:03.197223 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:03.197223 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:03.197223 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:03.197223 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 14:01:03.197223 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:01:03.197223 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 14:01:03.197223 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:03.197223 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 454/885] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 12 14:01:03 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:03.203373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78cf4c5-3e28-4b9d-8cc2-21908e350c9b None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:03.259029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 432/886] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:03 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 14:01:03.289450 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:03.312672 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78cf4c5-3e28-4b9d-8cc2-21908e350c9b None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:03.352279 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a4eb7-b194-441a-9156-74c0e0e8ebb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:03.352887 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a4eb7-b194-441a-9156-74c0e0e8ebb6 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:03.353162 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a4eb7-b194-441a-9156-74c0e0e8ebb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:03.353428 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a4eb7-b194-441a-9156-74c0e0e8ebb6 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:03.354065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a4eb7-b194-441a-9156-74c0e0e8ebb6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:03.354360 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a4eb7-b194-441a-9156-74c0e0e8ebb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:03.355727 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a4eb7-b194-441a-9156-74c0e0e8ebb6 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:03.360749 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 455/887] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 12 14:01:03 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:01:03.366382 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6393691c-dbd3-41f7-9208-3ce007280b83 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:03.479028 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6393691c-dbd3-41f7-9208-3ce007280b83 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:03.514249 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d872830-c472-4a70-990b-787981a25499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:03.514871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d872830-c472-4a70-990b-787981a25499 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:03.515175 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d872830-c472-4a70-990b-787981a25499 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:03.515443 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d872830-c472-4a70-990b-787981a25499 None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:03.516068 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d872830-c472-4a70-990b-787981a25499 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:03.516361 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d872830-c472-4a70-990b-787981a25499 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:03.517701 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d872830-c472-4a70-990b-787981a25499 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:03.523239 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 433/888] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 12 14:01:03 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:03.528932 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a4eb7-b194-441a-9156-74c0e0e8ebb6 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:03.548347 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d872830-c472-4a70-990b-787981a25499 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:03.551393 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:03.649992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a4eb7-b194-441a-9156-74c0e0e8ebb6 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:03.679855 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d872830-c472-4a70-990b-787981a25499 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:03.685631 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8627be-d200-433b-8b4a-d305f3505158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:03.686521 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8627be-d200-433b-8b4a-d305f3505158 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:03.687036 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8627be-d200-433b-8b4a-d305f3505158 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:03.687508 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8627be-d200-433b-8b4a-d305f3505158 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:03.688392 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8627be-d200-433b-8b4a-d305f3505158 None admin] RBAC: Adding query filter params (interface=public, service_id=37fd92f0afbc4cec8b03821671780675, region_id=RegionOne) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:03.688920 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8627be-d200-433b-8b4a-d305f3505158 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:03.690457 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8627be-d200-433b-8b4a-d305f3505158 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:03.696154 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 456/889] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Sep 12 14:01:03 2022] GET /identity/v3/endpoints?service_id=37fd92f0afbc4cec8b03821671780675&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:03.701652 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8627be-d200-433b-8b4a-d305f3505158 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:03.710605 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:03.726554 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b23024-0949-4ce6-8ecd-ced7ea22eaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:03.727578 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b23024-0949-4ce6-8ecd-ced7ea22eaf0 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:03.728076 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b23024-0949-4ce6-8ecd-ced7ea22eaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:03.728582 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b23024-0949-4ce6-8ecd-ced7ea22eaf0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:03.739829 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b23024-0949-4ce6-8ecd-ced7ea22eaf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:01:03.745880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-50b23024-0949-4ce6-8ecd-ced7ea22eaf0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 14:01:03.745880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:03.745880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:03.745880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:03.745880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:03.745880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:03.745880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:03.745880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:03.745880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:03.745880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:03.745880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:03.745880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:03.745880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:01:03.745880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:01:03.745880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:01:03.745880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:01:03.746887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:01:03.746887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:01:03.746887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:01:03.746887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:01:03.746887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:03.746887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:03.746887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:03.746887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:03.746887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:03.746887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:03.746887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:03.746887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:03.746887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:03.746887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:03.746887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:03.746887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:03.747742 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:03.747742 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:03.747742 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:03.747742 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:03.747742 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:01:03.747742 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:01:03.747742 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:01:03.747742 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:01:03.747742 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:01:03.747742 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:01:03.747742 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 14:01:03.747742 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:03.750370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 434/890] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Sep 12 14:01:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:01:03.756202 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b23024-0949-4ce6-8ecd-ced7ea22eaf0 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:03.815737 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8627be-d200-433b-8b4a-d305f3505158 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:03.824025 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:03.853742 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2593df5e-4cf4-485c-97da-70908d2434dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:03.854692 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2593df5e-4cf4-485c-97da-70908d2434dc None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:03.855216 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2593df5e-4cf4-485c-97da-70908d2434dc None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:03.855704 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2593df5e-4cf4-485c-97da-70908d2434dc None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:03.856603 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2593df5e-4cf4-485c-97da-70908d2434dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:03.858174 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2593df5e-4cf4-485c-97da-70908d2434dc None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:03.864349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 457/891] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Sep 12 14:01:03 2022] GET /identity/v3/services => generated 1268 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:01:03.868095 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b23024-0949-4ce6-8ecd-ced7ea22eaf0 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:03.911552 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ed544c-a98e-4bc9-8760-b89ef9c8cad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:03.912476 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed544c-a98e-4bc9-8760-b89ef9c8cad7 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:03.912902 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed544c-a98e-4bc9-8760-b89ef9c8cad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:03.913968 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed544c-a98e-4bc9-8760-b89ef9c8cad7 None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:03.914826 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ed544c-a98e-4bc9-8760-b89ef9c8cad7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:03.915314 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ed544c-a98e-4bc9-8760-b89ef9c8cad7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:03.916885 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ed544c-a98e-4bc9-8760-b89ef9c8cad7 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:03.931334 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 435/892] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Sep 12 14:01:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:01:03.933538 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2593df5e-4cf4-485c-97da-70908d2434dc None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:04.013611 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4bfb4-c18e-43a9-b234-ddbebcd07f9a None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:04.013944 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4bfb4-c18e-43a9-b234-ddbebcd07f9a None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:04.014157 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4bfb4-c18e-43a9-b234-ddbebcd07f9a None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:04.015938 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:04.016988 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 436/893] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 14:01:04.020986 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fa2ed-0348-4f5a-9ed2-2f7e20c2be74 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:04.021262 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fa2ed-0348-4f5a-9ed2-2f7e20c2be74 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:04.021526 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fa2ed-0348-4f5a-9ed2-2f7e20c2be74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:04.042519 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-3f6fa2ed-0348-4f5a-9ed2-2f7e20c2be74 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:04.043755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-3f6fa2ed-0348-4f5a-9ed2-2f7e20c2be74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:04.055225 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2593df5e-4cf4-485c-97da-70908d2434dc None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:04.091988 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bee6f2-e74a-40bf-be3c-df2c64465f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:04.092973 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bee6f2-e74a-40bf-be3c-df2c64465f57 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:04.093253 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bee6f2-e74a-40bf-be3c-df2c64465f57 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:04.093522 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bee6f2-e74a-40bf-be3c-df2c64465f57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:04.105038 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bee6f2-e74a-40bf-be3c-df2c64465f57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:04.106619 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bee6f2-e74a-40bf-be3c-df2c64465f57 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:04.121801 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 437/894] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:04 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:01:04.122696 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-b0bee6f2-e74a-40bf-be3c-df2c64465f57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:01:04.122696 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:04.122696 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:04.122696 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:04.122696 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:04.122696 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:04.122696 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:04.122696 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:04.122696 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:04.122696 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:04.122696 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:04.122696 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:04.122696 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:01:04.122696 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:01:04.122696 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:01:04.122696 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:01:04.123845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:04.123845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:04.123845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:04.123845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:04.123845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:04.123845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:04.123845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:04.123845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:04.123845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:04.123845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:04.123845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:04.123845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:04.123845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:04.123845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:04.123845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:04.123845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:04.123845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:01:04.124776 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:01:04.124776 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:01:04.124776 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:04.124776 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 458/895] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:01:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:04.129490 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6fa2ed-0348-4f5a-9ed2-2f7e20c2be74 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:04.186351 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:04.237402 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6fa2ed-0348-4f5a-9ed2-2f7e20c2be74 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:04.274070 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc211e-791f-4d52-b208-66a6461e99bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:04.275447 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc211e-791f-4d52-b208-66a6461e99bc None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:04.275984 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc211e-791f-4d52-b208-66a6461e99bc None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:04.276448 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc211e-791f-4d52-b208-66a6461e99bc None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:04.277319 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fc211e-791f-4d52-b208-66a6461e99bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:04.277815 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fc211e-791f-4d52-b208-66a6461e99bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:04.279403 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fc211e-791f-4d52-b208-66a6461e99bc None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:04.291720 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 438/896] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:01:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:01:04.297134 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bee6f2-e74a-40bf-be3c-df2c64465f57 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:04.399206 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bee6f2-e74a-40bf-be3c-df2c64465f57 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:04.421962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:04.425321 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc211e-791f-4d52-b208-66a6461e99bc None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:04.434705 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8227086d-7068-4ece-b7a2-433d9b6d4762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:04.435362 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227086d-7068-4ece-b7a2-433d9b6d4762 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:04.435637 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227086d-7068-4ece-b7a2-433d9b6d4762 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:04.435918 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227086d-7068-4ece-b7a2-433d9b6d4762 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:04.444082 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8227086d-7068-4ece-b7a2-433d9b6d4762 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:04.446137 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8227086d-7068-4ece-b7a2-433d9b6d4762 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:04.462528 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-8227086d-7068-4ece-b7a2-433d9b6d4762 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 14:01:04.462528 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:04.462528 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:04.462528 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:04.462528 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:04.462528 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:04.462528 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:04.462528 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:04.462528 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:04.462528 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:04.462528 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:04.462528 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:04.462528 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:01:04.462528 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:01:04.462528 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:01:04.462528 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:01:04.463583 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:04.463583 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:04.463583 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:01:04.463583 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:01:04.463583 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:01:04.463583 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:01:04.463583 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:04.463583 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:04.463583 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:04.463583 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:04.463583 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:04.463583 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:04.463583 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:04.463583 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:04.463583 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:04.463583 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:04.463583 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:04.463583 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:04.464543 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:04.464543 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:04.464543 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:01:04.464543 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:01:04.464543 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:01:04.464543 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:01:04.464543 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:01:04.464543 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:01:04.464543 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 14:01:04.464543 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:04.464543 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 459/897] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Sep 12 14:01:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:04.469597 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8227086d-7068-4ece-b7a2-433d9b6d4762 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:04.556896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc211e-791f-4d52-b208-66a6461e99bc None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:04.575474 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:04.589665 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8227086d-7068-4ece-b7a2-433d9b6d4762 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:04.600026 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b84b91-324b-42a1-948b-b0309160d52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:04.600359 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b84b91-324b-42a1-948b-b0309160d52a None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:04.600636 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b84b91-324b-42a1-948b-b0309160d52a None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:04.600931 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b84b91-324b-42a1-948b-b0309160d52a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:04.601509 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b84b91-324b-42a1-948b-b0309160d52a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:04.603120 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b84b91-324b-42a1-948b-b0309160d52a None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:04.612814 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-81b84b91-324b-42a1-948b-b0309160d52a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 14:01:04.612814 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:04.612814 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 14:01:04.612814 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 14:01:04.612814 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 14:01:04.612814 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 14:01:04.612814 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 14:01:04.612814 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:01:04.612814 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 14:01:04.612814 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:04.612814 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 14:01:04.612814 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:04.612814 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:04.612814 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:04.612814 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:04.612814 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:04.613851 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:04.613851 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:04.613851 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:04.613851 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:04.613851 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:04.613851 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:04.613851 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:04.613851 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 14:01:04.613851 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 14:01:04.613851 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 14:01:04.613851 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 14:01:04.613851 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:04.613851 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:04.613851 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:04.613851 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:04.613851 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:04.613851 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:04.614740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:04.614740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:04.614740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:04.614740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:04.614740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:04.614740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:04.614740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:04.614740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:04.614740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:04.614740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:04.614740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 14:01:04.614740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:01:04.614740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 14:01:04.614740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:04.614740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 439/898] 10.222.0.42 () {56 vars in 1132 bytes} [Mon Sep 12 14:01:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:04.621703 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b84b91-324b-42a1-948b-b0309160d52a None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:04.636775 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b33fe-050e-4f52-922a-1fa319925316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:04.637393 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b33fe-050e-4f52-922a-1fa319925316 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:04.637669 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b33fe-050e-4f52-922a-1fa319925316 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:04.637923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b33fe-050e-4f52-922a-1fa319925316 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:04.638641 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70b33fe-050e-4f52-922a-1fa319925316 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:04.639026 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70b33fe-050e-4f52-922a-1fa319925316 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:04.640506 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70b33fe-050e-4f52-922a-1fa319925316 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:04.662992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 460/899] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Sep 12 14:01:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:04.664387 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b33fe-050e-4f52-922a-1fa319925316 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:04.709213 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:04.758059 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b84b91-324b-42a1-948b-b0309160d52a None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:04.772043 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b33fe-050e-4f52-922a-1fa319925316 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:04.806509 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd393dd-d3af-46ef-ae73-925d05f08e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:04.807743 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd393dd-d3af-46ef-ae73-925d05f08e6a None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:04.808126 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f7f74-6375-462a-8fe6-67bb38991404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:04.808487 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd393dd-d3af-46ef-ae73-925d05f08e6a None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:04.808740 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f7f74-6375-462a-8fe6-67bb38991404 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:04.809016 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f7f74-6375-462a-8fe6-67bb38991404 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:04.809284 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f7f74-6375-462a-8fe6-67bb38991404 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:04.809588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd393dd-d3af-46ef-ae73-925d05f08e6a None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:04.810536 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd393dd-d3af-46ef-ae73-925d05f08e6a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:04.811084 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd393dd-d3af-46ef-ae73-925d05f08e6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:04.813004 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd393dd-d3af-46ef-ae73-925d05f08e6a None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:04.832674 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 440/900] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Sep 12 14:01:04 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:01:04.833927 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd393dd-d3af-46ef-ae73-925d05f08e6a None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:04.889687 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:04.892732 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f7f74-6375-462a-8fe6-67bb38991404 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:04.895434 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f7f74-6375-462a-8fe6-67bb38991404 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:04.944174 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:04.962436 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd393dd-d3af-46ef-ae73-925d05f08e6a None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:04.999833 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603551d7-be82-464f-9f44-b4446841f563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:05.000797 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603551d7-be82-464f-9f44-b4446841f563 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:05.001271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603551d7-be82-464f-9f44-b4446841f563 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:05.001787 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603551d7-be82-464f-9f44-b4446841f563 None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:05.002598 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603551d7-be82-464f-9f44-b4446841f563 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:05.003107 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603551d7-be82-464f-9f44-b4446841f563 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:05.004726 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603551d7-be82-464f-9f44-b4446841f563 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:05.012399 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 441/901] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Sep 12 14:01:04 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:05.017880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603551d7-be82-464f-9f44-b4446841f563 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:05.030138 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 461/902] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:01:04 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 14:01:05.031643 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f7f74-6375-462a-8fe6-67bb38991404 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:05.138146 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f7f74-6375-462a-8fe6-67bb38991404 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:05.168470 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603551d7-be82-464f-9f44-b4446841f563 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:05.177958 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97757f3-7977-48bd-a49c-9ad6c87907e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:05.178716 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97757f3-7977-48bd-a49c-9ad6c87907e6 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:05.179759 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97757f3-7977-48bd-a49c-9ad6c87907e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:05.180095 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97757f3-7977-48bd-a49c-9ad6c87907e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:05.181480 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:05.190765 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:05.193776 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97757f3-7977-48bd-a49c-9ad6c87907e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:05.195705 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97757f3-7977-48bd-a49c-9ad6c87907e6 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:05.213056 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-f97757f3-7977-48bd-a49c-9ad6c87907e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:01:05.213056 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:05.213056 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:05.213056 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:05.213056 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:05.213056 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:05.213056 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:05.213056 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:05.213056 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:05.213056 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:05.213056 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:05.213056 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:05.213056 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:01:05.213056 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:01:05.213056 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:01:05.213056 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:01:05.214050 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:05.214050 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:05.214050 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:05.214050 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:05.214050 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:05.214050 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:05.214050 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:05.214050 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:05.214050 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:05.214050 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:05.214050 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:05.214050 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:05.214050 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:05.214050 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:05.214050 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:05.214050 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:05.214050 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:01:05.214953 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:01:05.214953 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:01:05.214953 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:05.219035 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 462/903] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:01:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:05.219645 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c9fd8-38a9-4abb-8b04-97f04bfb1306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:05.220394 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97757f3-7977-48bd-a49c-9ad6c87907e6 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:05.220639 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c9fd8-38a9-4abb-8b04-97f04bfb1306 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:05.221177 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c9fd8-38a9-4abb-8b04-97f04bfb1306 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:05.221646 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c9fd8-38a9-4abb-8b04-97f04bfb1306 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:05.222519 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278c9fd8-38a9-4abb-8b04-97f04bfb1306 None admin] RBAC: Adding query filter params (interface=public, service_id=604a339f3c0b41858e78ecf4f0ebef42, region_id=RegionOne) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:05.223035 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278c9fd8-38a9-4abb-8b04-97f04bfb1306 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:05.226363 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278c9fd8-38a9-4abb-8b04-97f04bfb1306 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:05.237534 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 442/904] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Sep 12 14:01:05 2022] GET /identity/v3/endpoints?service_id=604a339f3c0b41858e78ecf4f0ebef42&interface=public®ion_id=RegionOne => generated 194 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:01:05.238750 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c9fd8-38a9-4abb-8b04-97f04bfb1306 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:05.335883 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97757f3-7977-48bd-a49c-9ad6c87907e6 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:05.348232 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c9fd8-38a9-4abb-8b04-97f04bfb1306 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:05.376665 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576fd3b5-8b15-4495-8360-77356b5f2415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:05.377567 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576fd3b5-8b15-4495-8360-77356b5f2415 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:05.377976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576fd3b5-8b15-4495-8360-77356b5f2415 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:05.378374 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576fd3b5-8b15-4495-8360-77356b5f2415 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:05.379047 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576fd3b5-8b15-4495-8360-77356b5f2415 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:05.379347 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576fd3b5-8b15-4495-8360-77356b5f2415 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:05.380708 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576fd3b5-8b15-4495-8360-77356b5f2415 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:05.381783 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd596e5-170b-4195-928b-4b9398cc1713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:05.382446 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd596e5-170b-4195-928b-4b9398cc1713 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:05.382719 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd596e5-170b-4195-928b-4b9398cc1713 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:05.383021 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd596e5-170b-4195-928b-4b9398cc1713 None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:05.383608 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd596e5-170b-4195-928b-4b9398cc1713 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:05.384937 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd596e5-170b-4195-928b-4b9398cc1713 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:05.390598 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 443/905] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Sep 12 14:01:05 2022] GET /identity/v3/services => generated 1268 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:01:05.391769 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedffa44-f65d-4af8-bc0c-7eb981911cf9 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:05.392045 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedffa44-f65d-4af8-bc0c-7eb981911cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:05.392309 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedffa44-f65d-4af8-bc0c-7eb981911cf9 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:05.393654 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 444/906] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:01:05.399905 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe92f76-86d8-413d-8845-24335619a2f3 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:05.400176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe92f76-86d8-413d-8845-24335619a2f3 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:05.400445 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe92f76-86d8-413d-8845-24335619a2f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:05.403113 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 463/907] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:01:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:05.408703 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576fd3b5-8b15-4495-8360-77356b5f2415 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:05.424468 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-bbe92f76-86d8-413d-8845-24335619a2f3 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:05.425678 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-bbe92f76-86d8-413d-8845-24335619a2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:05.456545 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:05.483494 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:05.515503 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 445/908] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:05 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 14:01:05.539191 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576fd3b5-8b15-4495-8360-77356b5f2415 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:05.576763 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c691eba-001e-439c-ac29-692e56a20f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:05.577641 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c691eba-001e-439c-ac29-692e56a20f3b None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:05.578060 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c691eba-001e-439c-ac29-692e56a20f3b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:05.578483 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c691eba-001e-439c-ac29-692e56a20f3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:05.586415 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c691eba-001e-439c-ac29-692e56a20f3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:05.588046 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c691eba-001e-439c-ac29-692e56a20f3b None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:05.599453 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-0c691eba-001e-439c-ac29-692e56a20f3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:01:05.599453 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:05.599453 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:05.599453 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:05.599453 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:05.599453 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:05.599453 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:05.599453 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:05.599453 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:05.599453 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:05.599453 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:05.599453 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:05.599453 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:01:05.599453 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:01:05.599453 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:01:05.600517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:01:05.600517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:05.600517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:05.600517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:05.600517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:05.600517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:05.600517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:05.600517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:05.600517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:05.600517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:05.600517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:05.600517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:05.600517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:05.600517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:05.600517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:05.600517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:05.600517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:05.602235 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:01:05.602235 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:01:05.602235 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:01:05.602235 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:01:05.602235 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:01:05.602235 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:01:05.602235 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:01:05.602235 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:05.604290 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 464/909] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:01:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:05.610001 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe92f76-86d8-413d-8845-24335619a2f3 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:05.715258 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe92f76-86d8-413d-8845-24335619a2f3 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:05.750031 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9cf07-35eb-4439-9921-2f620366bbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:05.751074 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9cf07-35eb-4439-9921-2f620366bbf2 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:05.751525 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9cf07-35eb-4439-9921-2f620366bbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:05.751964 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9cf07-35eb-4439-9921-2f620366bbf2 None admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:05.752837 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9cf07-35eb-4439-9921-2f620366bbf2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:05.753319 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9cf07-35eb-4439-9921-2f620366bbf2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:05.754920 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9cf07-35eb-4439-9921-2f620366bbf2 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:05.768048 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 446/910] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 12 14:01:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:05.777102 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c691eba-001e-439c-ac29-692e56a20f3b None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:05.798191 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9cf07-35eb-4439-9921-2f620366bbf2 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:05.830967 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:05.886558 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c691eba-001e-439c-ac29-692e56a20f3b None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:05.903305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:05.929292 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a19f55-8e96-4c8b-b163-e7b4371ad6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:05.929902 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a19f55-8e96-4c8b-b163-e7b4371ad6bd None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:05.930183 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a19f55-8e96-4c8b-b163-e7b4371ad6bd None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:05.930455 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a19f55-8e96-4c8b-b163-e7b4371ad6bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:05.930974 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9cf07-35eb-4439-9921-2f620366bbf2 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:05.931213 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a19f55-8e96-4c8b-b163-e7b4371ad6bd None admin] RBAC: Adding query filter params (role.id=c5ed025905434cacbeed08d80fc31213, scope.project.id=f666a617f8d14eb18bbeb076032b82f2, user.id=1405e758a3af4e8ba659f498cd28ace1) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:05.931521 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a19f55-8e96-4c8b-b163-e7b4371ad6bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:05.933215 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a19f55-8e96-4c8b-b163-e7b4371ad6bd None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:05.940835 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 465/911] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 14:01:05 2022] GET /identity/v3/role_assignments?user.id=1405e758a3af4e8ba659f498cd28ace1&scope.project.id=f666a617f8d14eb18bbeb076032b82f2&role.id=c5ed025905434cacbeed08d80fc31213 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:01:05.974596 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c17223-57f4-433f-a577-64fee35bf784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:05.975169 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c17223-57f4-433f-a577-64fee35bf784 None admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:05.975445 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c17223-57f4-433f-a577-64fee35bf784 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:05.975748 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c17223-57f4-433f-a577-64fee35bf784 None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:05.976429 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c17223-57f4-433f-a577-64fee35bf784 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:05.977831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c17223-57f4-433f-a577-64fee35bf784 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:05.996961 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 447/912] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Sep 12 14:01:05 2022] POST /identity/v3/services => generated 241 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:01:06.129460 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea612b-dd76-4a0a-a776-409c49895459 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:06.129777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea612b-dd76-4a0a-a776-409c49895459 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:06.130025 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea612b-dd76-4a0a-a776-409c49895459 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:06.131756 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 466/913] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:01:06.138466 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba780463-1bc2-44bd-b6ef-1a44c816a7e2 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:06.138802 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba780463-1bc2-44bd-b6ef-1a44c816a7e2 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:06.139045 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba780463-1bc2-44bd-b6ef-1a44c816a7e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:06.159806 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-ba780463-1bc2-44bd-b6ef-1a44c816a7e2 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:06.161479 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-ba780463-1bc2-44bd-b6ef-1a44c816a7e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:06.238897 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 448/914] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:01:06.510152 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea612b-dd76-4a0a-a776-409c49895459 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:06.622593 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea612b-dd76-4a0a-a776-409c49895459 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:06.658161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835fbca9-3f98-4297-ac72-5724d0a5ca5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:06.658784 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835fbca9-3f98-4297-ac72-5724d0a5ca5d None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:06.659086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835fbca9-3f98-4297-ac72-5724d0a5ca5d None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:06.659354 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835fbca9-3f98-4297-ac72-5724d0a5ca5d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:06.659931 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835fbca9-3f98-4297-ac72-5724d0a5ca5d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:06.661266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835fbca9-3f98-4297-ac72-5724d0a5ca5d None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:06.670190 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-835fbca9-3f98-4297-ac72-5724d0a5ca5d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 14:01:06.670190 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:06.670190 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 14:01:06.670190 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 14:01:06.670190 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 14:01:06.670190 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 14:01:06.670190 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 14:01:06.670190 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:01:06.670190 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 14:01:06.670190 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:06.670190 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 14:01:06.670190 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:06.670190 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:06.670190 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:06.670190 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:06.670190 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:06.670190 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:06.671241 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:06.671241 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:06.671241 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:06.671241 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:06.671241 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:06.671241 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:06.671241 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 14:01:06.671241 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 14:01:06.671241 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 14:01:06.671241 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 14:01:06.671241 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:06.671241 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:06.671241 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:06.671241 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:06.671241 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:06.671241 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:06.672066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:06.672066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:06.672066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:06.672066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:06.672066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:06.672066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:06.672066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:06.672066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:06.672066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:06.672066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:06.672066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 14:01:06.672066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:01:06.672066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 14:01:06.672066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:06.672066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 467/915] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 12 14:01:06 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:06.679406 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba780463-1bc2-44bd-b6ef-1a44c816a7e2 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:06.692053 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:06.793546 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba780463-1bc2-44bd-b6ef-1a44c816a7e2 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:06.829568 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7823817-e7da-4002-86e0-f4adc8215a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:06.830176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7823817-e7da-4002-86e0-f4adc8215a10 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:06.830454 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7823817-e7da-4002-86e0-f4adc8215a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:06.830724 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7823817-e7da-4002-86e0-f4adc8215a10 None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:06.831400 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7823817-e7da-4002-86e0-f4adc8215a10 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:06.831699 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7823817-e7da-4002-86e0-f4adc8215a10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:06.833056 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7823817-e7da-4002-86e0-f4adc8215a10 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:06.838221 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 449/916] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 12 14:01:06 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:06.843787 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835fbca9-3f98-4297-ac72-5724d0a5ca5d None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:06.890005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:06.954447 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835fbca9-3f98-4297-ac72-5724d0a5ca5d None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:06.989315 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721643db-228b-447f-90ef-a506f28b9229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:06.990219 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721643db-228b-447f-90ef-a506f28b9229 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:06.990703 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721643db-228b-447f-90ef-a506f28b9229 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:06.991216 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721643db-228b-447f-90ef-a506f28b9229 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:06.992578 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721643db-228b-447f-90ef-a506f28b9229 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:06.993097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721643db-228b-447f-90ef-a506f28b9229 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:06.994635 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721643db-228b-447f-90ef-a506f28b9229 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:07.000082 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 468/917] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 12 14:01:06 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:07.005933 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7823817-e7da-4002-86e0-f4adc8215a10 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:07.088331 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:07.114141 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7823817-e7da-4002-86e0-f4adc8215a10 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:07.150571 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f9ec9-9a73-479c-9947-88adfb71e5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:07.151206 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f9ec9-9a73-479c-9947-88adfb71e5b8 None admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:07.151524 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f9ec9-9a73-479c-9947-88adfb71e5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:07.151800 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f9ec9-9a73-479c-9947-88adfb71e5b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:07.152474 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f9ec9-9a73-479c-9947-88adfb71e5b8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:07.153866 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f9ec9-9a73-479c-9947-88adfb71e5b8 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:07.178117 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 450/918] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:01:07 2022] POST /identity/v3/endpoints => generated 346 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:01:07.718875 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bb55d-5eee-4fd6-b60b-f993c39610b4 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:07.719403 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bb55d-5eee-4fd6-b60b-f993c39610b4 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:07.719711 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bb55d-5eee-4fd6-b60b-f993c39610b4 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:07.721415 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 469/919] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:01:07.727472 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac5c6a9-ff92-4c0e-b0ca-964250ef22ec None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:07.727891 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac5c6a9-ff92-4c0e-b0ca-964250ef22ec None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:07.728195 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac5c6a9-ff92-4c0e-b0ca-964250ef22ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:07.746587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-7ac5c6a9-ff92-4c0e-b0ca-964250ef22ec None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:07.747818 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-7ac5c6a9-ff92-4c0e-b0ca-964250ef22ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:07.823585 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 451/920] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:01:08.097911 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021bb55d-5eee-4fd6-b60b-f993c39610b4 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:08.216566 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021bb55d-5eee-4fd6-b60b-f993c39610b4 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:08.218920 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:08.243575 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5faa2f-89e4-4cc5-a82f-602b0a0a9bc3 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:08.244077 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5faa2f-89e4-4cc5-a82f-602b0a0a9bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:08.244489 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5faa2f-89e4-4cc5-a82f-602b0a0a9bc3 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:08.246204 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 452/921] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:01:08.251803 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa6a1d-3d4e-4294-9f82-2c32d1e6474a None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:08.252075 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa6a1d-3d4e-4294-9f82-2c32d1e6474a None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:08.252338 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa6a1d-3d4e-4294-9f82-2c32d1e6474a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:08.254622 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0f14c-a7cc-4d71-a1a9-e0558a7822dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:08.255303 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0f14c-a7cc-4d71-a1a9-e0558a7822dd None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:08.255580 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0f14c-a7cc-4d71-a1a9-e0558a7822dd None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:08.255848 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0f14c-a7cc-4d71-a1a9-e0558a7822dd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:08.256426 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0f14c-a7cc-4d71-a1a9-e0558a7822dd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:08.257816 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0f14c-a7cc-4d71-a1a9-e0558a7822dd None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:08.266667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-85d0f14c-a7cc-4d71-a1a9-e0558a7822dd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 14:01:08.266667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:08.266667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 14:01:08.266667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 14:01:08.266667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 14:01:08.266667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 14:01:08.266667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 14:01:08.266667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:01:08.266667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 14:01:08.266667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:08.266667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 14:01:08.266667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:08.266667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:08.266667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:08.266667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:08.266667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:08.267740 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:08.267740 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:08.267740 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:08.267740 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:08.267740 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:08.267740 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:08.267740 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:08.267740 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 14:01:08.267740 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 14:01:08.267740 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 14:01:08.267740 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 14:01:08.267740 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:08.267740 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:08.267740 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:08.267740 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:08.267740 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:08.267740 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:08.268622 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:08.268622 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:08.268622 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:08.268622 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:08.268622 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:08.268622 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:08.268622 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:08.268622 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:08.268622 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:08.268622 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:08.268622 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 14:01:08.268622 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:01:08.268622 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 14:01:08.268622 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:08.268622 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 470/922] 10.222.0.42 () {56 vars in 1132 bytes} [Mon Sep 12 14:01:08 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:08.269397 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0487ed8-c2f2-4dec-a84d-815cbcd2fc93 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:08.269463 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0487ed8-c2f2-4dec-a84d-815cbcd2fc93 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:08.270090 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0487ed8-c2f2-4dec-a84d-815cbcd2fc93 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:08.278573 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-47aa6a1d-3d4e-4294-9f82-2c32d1e6474a None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:08.278727 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 471/923] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:01:08.278827 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-47aa6a1d-3d4e-4294-9f82-2c32d1e6474a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:08.278931 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0487ed8-c2f2-4dec-a84d-815cbcd2fc93 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:08.326756 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:08.347005 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 453/924] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:01:08.348614 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c9e12-a6be-444a-9d64-6fcc1122558c None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:08.348825 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c9e12-a6be-444a-9d64-6fcc1122558c None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:08.349093 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c9e12-a6be-444a-9d64-6fcc1122558c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:08.367150 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-ec0c9e12-a6be-444a-9d64-6fcc1122558c None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:08.368327 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-ec0c9e12-a6be-444a-9d64-6fcc1122558c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:08.394811 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0487ed8-c2f2-4dec-a84d-815cbcd2fc93 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:08.441519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e93dac-e595-4300-957e-e9391ae0adb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:08.442129 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e93dac-e595-4300-957e-e9391ae0adb3 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:08.442410 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e93dac-e595-4300-957e-e9391ae0adb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:08.442683 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e93dac-e595-4300-957e-e9391ae0adb3 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:08.443365 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e93dac-e595-4300-957e-e9391ae0adb3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:08.443669 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e93dac-e595-4300-957e-e9391ae0adb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:08.445107 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e93dac-e595-4300-957e-e9391ae0adb3 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:08.447201 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 454/925] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:01:08.450739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 472/926] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Sep 12 14:01:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:01:08.456105 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c9e12-a6be-444a-9d64-6fcc1122558c None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:08.572988 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c9e12-a6be-444a-9d64-6fcc1122558c None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:08.589894 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:08.608935 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682c04a4-cbd5-4c9e-898d-e5df7e87cd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:08.609820 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c04a4-cbd5-4c9e-898d-e5df7e87cd03 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:08.610316 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c04a4-cbd5-4c9e-898d-e5df7e87cd03 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:08.610781 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c04a4-cbd5-4c9e-898d-e5df7e87cd03 None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:08.611644 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682c04a4-cbd5-4c9e-898d-e5df7e87cd03 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:08.612141 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682c04a4-cbd5-4c9e-898d-e5df7e87cd03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:08.613777 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682c04a4-cbd5-4c9e-898d-e5df7e87cd03 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:08.615090 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e93dac-e595-4300-957e-e9391ae0adb3 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:08.620258 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 455/927] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Sep 12 14:01:08 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:08.625357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682c04a4-cbd5-4c9e-898d-e5df7e87cd03 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:08.695192 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:08.733432 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682c04a4-cbd5-4c9e-898d-e5df7e87cd03 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:08.738603 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e93dac-e595-4300-957e-e9391ae0adb3 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:08.768172 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954409d8-9e6f-4ed5-bd3b-35991ed81f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:08.768770 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954409d8-9e6f-4ed5-bd3b-35991ed81f28 None admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:08.769045 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954409d8-9e6f-4ed5-bd3b-35991ed81f28 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:08.769311 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954409d8-9e6f-4ed5-bd3b-35991ed81f28 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:08.769959 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954409d8-9e6f-4ed5-bd3b-35991ed81f28 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:08.771332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954409d8-9e6f-4ed5-bd3b-35991ed81f28 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:08.775569 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb9a0f1-d373-4cbe-995e-b5c3cabb52b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:08.776191 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb9a0f1-d373-4cbe-995e-b5c3cabb52b5 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:08.776466 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb9a0f1-d373-4cbe-995e-b5c3cabb52b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:08.776731 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb9a0f1-d373-4cbe-995e-b5c3cabb52b5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:08.777326 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb9a0f1-d373-4cbe-995e-b5c3cabb52b5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:08.778653 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb9a0f1-d373-4cbe-995e-b5c3cabb52b5 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:08.790074 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-ecb9a0f1-d373-4cbe-995e-b5c3cabb52b5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 14:01:08.790074 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:08.790074 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 14:01:08.790074 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 14:01:08.790074 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 14:01:08.790074 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 14:01:08.790074 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 14:01:08.790074 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:01:08.790074 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 14:01:08.790074 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:08.790074 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 14:01:08.790074 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:08.790074 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:08.790074 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:08.790074 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:08.790074 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:08.790074 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:08.796435 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:08.796435 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:08.796435 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:08.796435 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:08.796435 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:08.796435 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:08.796435 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 14:01:08.796435 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 14:01:08.796435 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 14:01:08.796435 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 14:01:08.796435 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:08.796435 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:08.796435 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:08.796435 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:08.796435 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:08.796435 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:08.797307 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:08.797307 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:08.797307 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:08.797307 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:08.797307 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:08.797307 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:08.797307 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:08.797307 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:08.797307 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:08.797307 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:08.797307 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 14:01:08.797307 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:01:08.797307 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 14:01:08.797307 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:08.797307 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 473/928] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Sep 12 14:01:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:01:08.798181 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb9a0f1-d373-4cbe-995e-b5c3cabb52b5 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:08.803158 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 456/929] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:01:08 2022] POST /identity/v3/endpoints => generated 347 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:01:08.806971 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954409d8-9e6f-4ed5-bd3b-35991ed81f28 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:08.857792 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:08.931905 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:08.942123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb9a0f1-d373-4cbe-995e-b5c3cabb52b5 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:08.944135 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954409d8-9e6f-4ed5-bd3b-35991ed81f28 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:08.982773 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fb6e31-f42a-40c0-85c5-19dc2f45e548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:08.983440 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb6e31-f42a-40c0-85c5-19dc2f45e548 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:08.983746 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb6e31-f42a-40c0-85c5-19dc2f45e548 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:08.984018 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb6e31-f42a-40c0-85c5-19dc2f45e548 None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:08.984800 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fb6e31-f42a-40c0-85c5-19dc2f45e548 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:08.985096 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fb6e31-f42a-40c0-85c5-19dc2f45e548 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:08.986481 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fb6e31-f42a-40c0-85c5-19dc2f45e548 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:08.990636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f4658-6246-4508-b182-feace0dadb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:08.991309 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f4658-6246-4508-b182-feace0dadb05 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:08.991590 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f4658-6246-4508-b182-feace0dadb05 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:08.991863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f4658-6246-4508-b182-feace0dadb05 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:08.991991 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 457/930] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Sep 12 14:01:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:08.998287 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fb6e31-f42a-40c0-85c5-19dc2f45e548 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:09.008055 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f4658-6246-4508-b182-feace0dadb05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:01:09.014635 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-900f4658-6246-4508-b182-feace0dadb05 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 14:01:09.014635 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:09.014635 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:09.014635 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:09.014635 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:09.014635 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:09.014635 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:09.014635 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:09.014635 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:09.014635 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:09.014635 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:09.014635 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:09.014635 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:01:09.014635 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:01:09.014635 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:01:09.014635 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:01:09.015754 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:01:09.015754 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:01:09.015754 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:01:09.015754 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:01:09.015754 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:09.015754 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:09.015754 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:09.015754 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:09.015754 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:09.015754 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:09.015754 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:09.015754 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:09.015754 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:09.015754 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:09.015754 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:09.015754 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:09.016927 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:09.016927 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:09.016927 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:09.016927 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:09.016927 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:01:09.016927 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:01:09.016927 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:01:09.016927 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:01:09.016927 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:01:09.016927 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:01:09.016927 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 14:01:09.016927 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:09.016927 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 474/931] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Sep 12 14:01:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:09.024293 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f4658-6246-4508-b182-feace0dadb05 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:09.102173 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:09.109451 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fb6e31-f42a-40c0-85c5-19dc2f45e548 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:09.150030 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f4658-6246-4508-b182-feace0dadb05 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:09.153107 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a99dfb-b33b-4446-ab8e-cabf032241d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:09.153987 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a99dfb-b33b-4446-ab8e-cabf032241d5 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:09.154408 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a99dfb-b33b-4446-ab8e-cabf032241d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:09.154874 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a99dfb-b33b-4446-ab8e-cabf032241d5 None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:09.155662 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a99dfb-b33b-4446-ab8e-cabf032241d5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:09.156103 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a99dfb-b33b-4446-ab8e-cabf032241d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:09.158151 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a99dfb-b33b-4446-ab8e-cabf032241d5 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:09.165946 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 458/932] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Sep 12 14:01:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:01:09.172644 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a99dfb-b33b-4446-ab8e-cabf032241d5 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:09.194159 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac38a58e-1cb6-417c-8494-b5eb74b1ecc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:09.195412 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac38a58e-1cb6-417c-8494-b5eb74b1ecc3 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:09.195660 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac38a58e-1cb6-417c-8494-b5eb74b1ecc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:09.195928 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac38a58e-1cb6-417c-8494-b5eb74b1ecc3 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:09.196913 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac38a58e-1cb6-417c-8494-b5eb74b1ecc3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:09.197191 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac38a58e-1cb6-417c-8494-b5eb74b1ecc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:09.198532 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac38a58e-1cb6-417c-8494-b5eb74b1ecc3 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:09.212826 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 475/933] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Sep 12 14:01:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:01:09.215319 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac38a58e-1cb6-417c-8494-b5eb74b1ecc3 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:09.237897 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:09.315250 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a99dfb-b33b-4446-ab8e-cabf032241d5 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:09.335784 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac38a58e-1cb6-417c-8494-b5eb74b1ecc3 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:09.357684 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3d5a9-bbbb-4f4c-a8fb-407261d4ac04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:09.358757 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3d5a9-bbbb-4f4c-a8fb-407261d4ac04 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:09.359262 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3d5a9-bbbb-4f4c-a8fb-407261d4ac04 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:09.359722 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3d5a9-bbbb-4f4c-a8fb-407261d4ac04 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:09.360720 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3d5a9-bbbb-4f4c-a8fb-407261d4ac04 None admin] RBAC: Adding query filter params (interface=public, service_id=bd4911ae3aba42028c13a9ec4c66e893, region_id=RegionOne) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:09.361228 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3d5a9-bbbb-4f4c-a8fb-407261d4ac04 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:09.363203 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3d5a9-bbbb-4f4c-a8fb-407261d4ac04 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:09.370565 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 459/934] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Sep 12 14:01:09 2022] GET /identity/v3/endpoints?service_id=bd4911ae3aba42028c13a9ec4c66e893&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:09.378316 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3d5a9-bbbb-4f4c-a8fb-407261d4ac04 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:09.383118 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5588e-a78e-4749-9182-e777baf7eb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:09.383848 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5588e-a78e-4749-9182-e777baf7eb06 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:09.384132 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5588e-a78e-4749-9182-e777baf7eb06 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:09.384392 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5588e-a78e-4749-9182-e777baf7eb06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:09.401043 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df5588e-a78e-4749-9182-e777baf7eb06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:09.402996 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df5588e-a78e-4749-9182-e777baf7eb06 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:09.411271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:09.423966 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-4df5588e-a78e-4749-9182-e777baf7eb06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:01:09.423966 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:09.423966 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:09.423966 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:09.423966 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:09.423966 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:09.423966 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:09.423966 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:09.423966 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:09.423966 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:09.423966 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:09.423966 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:09.423966 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:01:09.423966 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:01:09.423966 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:01:09.423966 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:01:09.425027 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:09.425027 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:09.425027 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:09.425027 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:09.425027 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:09.425027 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:09.425027 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:09.425027 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:09.425027 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:09.425027 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:09.425027 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:09.425027 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:09.425027 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:09.425027 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:09.425027 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:09.425027 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:09.425027 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:01:09.425904 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:01:09.425904 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:01:09.425904 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:09.426641 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 476/935] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:01:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:09.436169 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5588e-a78e-4749-9182-e777baf7eb06 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:09.499537 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3d5a9-bbbb-4f4c-a8fb-407261d4ac04 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:09.524523 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:09.537062 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d0255c-c607-4c80-b3a9-cd14349596e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:09.537740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0255c-c607-4c80-b3a9-cd14349596e0 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:09.538041 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0255c-c607-4c80-b3a9-cd14349596e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:09.538319 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0255c-c607-4c80-b3a9-cd14349596e0 None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:09.538944 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d0255c-c607-4c80-b3a9-cd14349596e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:09.540291 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d0255c-c607-4c80-b3a9-cd14349596e0 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:09.550890 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 460/936] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Sep 12 14:01:09 2022] GET /identity/v3/services => generated 1497 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:01:09.550890 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce58ce2-54f8-4cfd-bcd6-7791e9b9eeda None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:09.550890 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce58ce2-54f8-4cfd-bcd6-7791e9b9eeda None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:09.550890 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce58ce2-54f8-4cfd-bcd6-7791e9b9eeda None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:09.550890 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 461/937] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:01:09.555219 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a567d8e-eb74-4b8b-8e43-96ac246c21b6 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:09.556713 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a567d8e-eb74-4b8b-8e43-96ac246c21b6 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:09.556713 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a567d8e-eb74-4b8b-8e43-96ac246c21b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:09.569188 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5588e-a78e-4749-9182-e777baf7eb06 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:09.584718 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-6a567d8e-eb74-4b8b-8e43-96ac246c21b6 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:09.587485 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-6a567d8e-eb74-4b8b-8e43-96ac246c21b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:09.615249 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee8c441-26c2-4265-a200-4e13aa8a38a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:09.615985 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8c441-26c2-4265-a200-4e13aa8a38a1 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:09.616359 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8c441-26c2-4265-a200-4e13aa8a38a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:09.616706 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8c441-26c2-4265-a200-4e13aa8a38a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:09.617449 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee8c441-26c2-4265-a200-4e13aa8a38a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:09.617824 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee8c441-26c2-4265-a200-4e13aa8a38a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:09.619343 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee8c441-26c2-4265-a200-4e13aa8a38a1 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:09.641570 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 477/938] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:01:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:01:09.643519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee8c441-26c2-4265-a200-4e13aa8a38a1 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:09.687070 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 462/939] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:01:09.768126 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee8c441-26c2-4265-a200-4e13aa8a38a1 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:09.799784 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:09.806792 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc7c09-f2c3-44bf-8c33-136323f56cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:09.807458 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc7c09-f2c3-44bf-8c33-136323f56cac None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:09.807734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc7c09-f2c3-44bf-8c33-136323f56cac None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:09.808000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc7c09-f2c3-44bf-8c33-136323f56cac None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:09.816020 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfc7c09-f2c3-44bf-8c33-136323f56cac None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:09.823871 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfc7c09-f2c3-44bf-8c33-136323f56cac None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:09.843863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-ddfc7c09-f2c3-44bf-8c33-136323f56cac None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 14:01:09.843863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:09.843863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:09.843863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:09.843863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:09.843863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:09.843863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:09.843863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:09.843863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:09.843863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:09.843863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:09.843863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:09.843863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:01:09.843863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:01:09.843863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:01:09.843863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:01:09.845041 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:09.845041 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:09.845041 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:01:09.845041 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:01:09.845041 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:01:09.845041 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:01:09.845041 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:09.845041 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:09.845041 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:09.845041 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:09.845041 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:09.845041 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:09.845041 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:09.845041 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:09.845041 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:09.845041 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:09.845041 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:09.845041 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:09.846106 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:09.846106 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:09.846106 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:01:09.846106 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:01:09.846106 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:01:09.846106 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:01:09.846106 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:01:09.846106 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:01:09.846106 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 14:01:09.846106 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:09.849830 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 478/940] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Sep 12 14:01:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:09.851345 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a567d8e-eb74-4b8b-8e43-96ac246c21b6 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:09.879235 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:09.963648 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a567d8e-eb74-4b8b-8e43-96ac246c21b6 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:10.002984 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc24308-cfc0-4cf3-ab34-86cdefb33a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:10.003828 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc24308-cfc0-4cf3-ab34-86cdefb33a7d None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:10.004285 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc24308-cfc0-4cf3-ab34-86cdefb33a7d None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:10.004715 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc24308-cfc0-4cf3-ab34-86cdefb33a7d None admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:10.005591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc24308-cfc0-4cf3-ab34-86cdefb33a7d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:10.006041 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc24308-cfc0-4cf3-ab34-86cdefb33a7d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:10.008364 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc24308-cfc0-4cf3-ab34-86cdefb33a7d None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:10.029374 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc7c09-f2c3-44bf-8c33-136323f56cac None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:10.029907 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 463/941] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Sep 12 14:01:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:10.035486 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc24308-cfc0-4cf3-ab34-86cdefb33a7d None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:10.136943 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:10.150616 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc7c09-f2c3-44bf-8c33-136323f56cac None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:10.169282 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc24308-cfc0-4cf3-ab34-86cdefb33a7d None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:10.189406 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9278f71-d3c3-491d-b947-ee22b6893867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:10.190277 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9278f71-d3c3-491d-b947-ee22b6893867 None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:10.190668 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9278f71-d3c3-491d-b947-ee22b6893867 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:10.191103 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9278f71-d3c3-491d-b947-ee22b6893867 None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:10.191853 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9278f71-d3c3-491d-b947-ee22b6893867 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:10.193471 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9278f71-d3c3-491d-b947-ee22b6893867 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:10.197178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: WARNING keystone.server.flask.common [None req-d9278f71-d3c3-491d-b947-ee22b6893867 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 12 14:01:10.209320 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18635b-fdda-485e-9ec8-783867539f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:10.210046 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18635b-fdda-485e-9ec8-783867539f26 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:10.210820 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18635b-fdda-485e-9ec8-783867539f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:10.211112 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18635b-fdda-485e-9ec8-783867539f26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:10.217185 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.sql.core [None req-d9278f71-d3c3-491d-b947-ee22b6893867 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 12 14:01:10.217185 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 12 14:01:10.217185 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [parameters: {'id': '48cf4f9c96604594b5f414b7b764544d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 12 14:01:10.217185 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81597) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 12 14:01:10.222605 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c18635b-fdda-485e-9ec8-783867539f26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:10.224065 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c18635b-fdda-485e-9ec8-783867539f26 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:10.225327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-d9278f71-d3c3-491d-b947-ee22b6893867 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 48cf4f9c96604594b5f414b7b764544d.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 48cf4f9c96604594b5f414b7b764544d. Sep 12 14:01:10.225327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:10.225327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 12 14:01:10.225327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 12 14:01:10.225327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 12 14:01:10.225327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 12 14:01:10.225327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 12 14:01:10.225327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application result = self._query(query) Sep 12 14:01:10.225327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 12 14:01:10.225327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application conn.query(q) Sep 12 14:01:10.225327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 12 14:01:10.225327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 12 14:01:10.225327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 12 14:01:10.226913 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application result.read() Sep 12 14:01:10.226913 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 12 14:01:10.226913 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 12 14:01:10.226913 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 12 14:01:10.226913 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 12 14:01:10.226913 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 12 14:01:10.226913 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 12 14:01:10.226913 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 12 14:01:10.226913 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 12 14:01:10.226913 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 12 14:01:10.226913 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:10.226913 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 12 14:01:10.226913 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:10.226913 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:10.226913 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 12 14:01:10.226913 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 12 14:01:10.226913 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 12 14:01:10.226913 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 12 14:01:10.229000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 12 14:01:10.229000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application next(self.gen) Sep 12 14:01:10.229000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 12 14:01:10.229000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application yield resource Sep 12 14:01:10.229000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 12 14:01:10.229000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application next(self.gen) Sep 12 14:01:10.229000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 12 14:01:10.229000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self.session.rollback() Sep 12 14:01:10.229000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 12 14:01:10.229000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self.force_reraise() Sep 12 14:01:10.229000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 12 14:01:10.229000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise self.value Sep 12 14:01:10.229000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 12 14:01:10.229000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 12 14:01:10.229000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 12 14:01:10.229000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application session.commit() Sep 12 14:01:10.229000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 12 14:01:10.230903 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 12 14:01:10.230903 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 12 14:01:10.230903 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._prepare_impl() Sep 12 14:01:10.230903 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 12 14:01:10.230903 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self.session.flush() Sep 12 14:01:10.230903 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 12 14:01:10.230903 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._flush(objects) Sep 12 14:01:10.230903 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 12 14:01:10.230903 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 12 14:01:10.230903 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 12 14:01:10.230903 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application compat.raise_( Sep 12 14:01:10.230903 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 12 14:01:10.230903 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception Sep 12 14:01:10.230903 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 12 14:01:10.230903 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application flush_context.execute() Sep 12 14:01:10.230903 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 12 14:01:10.230903 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rec.execute(self) Sep 12 14:01:10.232799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 12 14:01:10.232799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 12 14:01:10.232799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 12 14:01:10.232799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 12 14:01:10.232799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 12 14:01:10.232799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 12 14:01:10.232799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 12 14:01:10.232799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 12 14:01:10.232799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 12 14:01:10.232799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 12 14:01:10.232799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 12 14:01:10.232799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 12 14:01:10.232799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 12 14:01:10.232799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 12 14:01:10.232799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 12 14:01:10.243712 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 12 14:01:10.243712 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 12 14:01:10.243712 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception Sep 12 14:01:10.243712 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 12 14:01:10.243712 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 12 14:01:10.243712 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 12 14:01:10.243712 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 12 14:01:10.243712 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 12 14:01:10.243712 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application result = self._query(query) Sep 12 14:01:10.243712 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 12 14:01:10.243712 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application conn.query(q) Sep 12 14:01:10.243712 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 12 14:01:10.243712 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 12 14:01:10.243712 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 12 14:01:10.243712 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application result.read() Sep 12 14:01:10.243712 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 12 14:01:10.243712 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 12 14:01:10.244834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 12 14:01:10.244834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 12 14:01:10.244834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 12 14:01:10.244834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 12 14:01:10.244834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 12 14:01:10.244834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 12 14:01:10.244834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 12 14:01:10.244834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 12 14:01:10.244834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [parameters: {'id': '48cf4f9c96604594b5f414b7b764544d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 12 14:01:10.244834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 12 14:01:10.244834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:10.244834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 14:01:10.244834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:10.244834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:10.244834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 12 14:01:10.245850 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 12 14:01:10.245850 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 12 14:01:10.245850 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 12 14:01:10.245850 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 12 14:01:10.245850 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:10.245850 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 14:01:10.245850 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:10.245850 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:10.245850 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:10.245850 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:10.245850 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:10.245850 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:10.245850 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:10.245850 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:10.245850 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:10.245850 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:10.245850 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:10.247002 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:10.247002 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 12 14:01:10.247002 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 12 14:01:10.247002 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:10.247002 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:10.247002 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 12 14:01:10.247002 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 12 14:01:10.247002 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 48cf4f9c96604594b5f414b7b764544d. Sep 12 14:01:10.247002 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:10.247002 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 479/942] 10.222.0.42 () {60 vars in 1140 bytes} [Mon Sep 12 14:01:10 2022] POST /identity/v3/projects => generated 262 bytes in 199 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:01:10.247002 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9278f71-d3c3-491d-b947-ee22b6893867 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:10.247740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-5c18635b-fdda-485e-9ec8-783867539f26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:01:10.247740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:10.247740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:10.247740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:10.247740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:10.247740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:10.247740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:10.247740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:10.247740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:10.247740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:10.247740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:10.247740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:10.247740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:01:10.247740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:01:10.247740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:01:10.247740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:01:10.248793 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:10.248793 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:10.248793 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:10.248793 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:10.248793 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:10.248793 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:10.248793 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:10.248793 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:10.248793 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:10.248793 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:10.248793 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:10.248793 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:10.248793 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:10.248793 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:10.248793 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:10.248793 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:10.248793 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:01:10.249908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:01:10.249908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:01:10.249908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:10.254132 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 464/943] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:01:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:01:10.254132 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18635b-fdda-485e-9ec8-783867539f26 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:10.307551 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:10.371359 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18635b-fdda-485e-9ec8-783867539f26 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:10.378340 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9278f71-d3c3-491d-b947-ee22b6893867 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:10.401007 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:10.411063 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130e25c-4373-4307-b6ac-59ed80e04611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:10.412012 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130e25c-4373-4307-b6ac-59ed80e04611 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:10.412489 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130e25c-4373-4307-b6ac-59ed80e04611 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:10.412956 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130e25c-4373-4307-b6ac-59ed80e04611 None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:10.413962 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5130e25c-4373-4307-b6ac-59ed80e04611 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:10.414529 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5130e25c-4373-4307-b6ac-59ed80e04611 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:10.416150 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5130e25c-4373-4307-b6ac-59ed80e04611 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:10.422640 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382e4e7-d121-4171-bb43-9528df906cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:10.423350 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382e4e7-d121-4171-bb43-9528df906cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:10.423623 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382e4e7-d121-4171-bb43-9528df906cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:10.423889 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382e4e7-d121-4171-bb43-9528df906cd2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:10.432144 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3382e4e7-d121-4171-bb43-9528df906cd2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:10.433468 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 465/944] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:01:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:01:10.434255 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3382e4e7-d121-4171-bb43-9528df906cd2 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:10.438882 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130e25c-4373-4307-b6ac-59ed80e04611 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:10.450252 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-3382e4e7-d121-4171-bb43-9528df906cd2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:01:10.450252 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:10.450252 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:10.450252 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:10.450252 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:10.450252 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:10.450252 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:10.450252 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:10.450252 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:10.450252 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:10.450252 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:10.450252 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:10.450252 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:01:10.450252 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:01:10.450252 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:01:10.451172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:01:10.451172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:10.451172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:10.451172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:10.451172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:10.451172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:10.451172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:10.451172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:10.451172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:10.451172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:10.451172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:10.451172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:10.451172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:10.451172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:10.451172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:10.451172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:10.451172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:10.452065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:01:10.452065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:01:10.452065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:01:10.452065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:01:10.452065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:01:10.452065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:01:10.452065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:01:10.452065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:10.454042 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 480/945] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:01:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:01:10.462252 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382e4e7-d121-4171-bb43-9528df906cd2 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:10.556933 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130e25c-4373-4307-b6ac-59ed80e04611 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:10.592128 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382e4e7-d121-4171-bb43-9528df906cd2 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:10.599385 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b9648b-2522-44d3-a455-3cdfa39f8605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:10.600010 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9648b-2522-44d3-a455-3cdfa39f8605 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:10.600282 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9648b-2522-44d3-a455-3cdfa39f8605 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:10.600596 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9648b-2522-44d3-a455-3cdfa39f8605 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:10.602706 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:10.608440 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b9648b-2522-44d3-a455-3cdfa39f8605 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:10.610090 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b9648b-2522-44d3-a455-3cdfa39f8605 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:10.621573 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-85b9648b-2522-44d3-a455-3cdfa39f8605 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:01:10.621573 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:10.621573 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:10.621573 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:10.621573 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:10.621573 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:10.621573 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:10.621573 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:10.621573 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:10.621573 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:10.621573 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:10.621573 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:10.621573 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:01:10.621573 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:01:10.621573 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:01:10.622474 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:01:10.622474 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:10.622474 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:10.622474 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:10.622474 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:10.622474 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:10.622474 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:10.622474 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:10.622474 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:10.622474 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:10.622474 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:10.622474 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:10.622474 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:10.622474 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:10.622474 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:10.622474 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:10.622474 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:10.623403 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:01:10.623403 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:01:10.623403 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:01:10.623403 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:01:10.623403 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:01:10.623403 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:01:10.623403 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:01:10.623403 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:10.627537 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 466/946] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:01:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:01:10.628683 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b9648b-2522-44d3-a455-3cdfa39f8605 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:10.635356 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593764ca-f929-4458-b700-197318ffc50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:10.636212 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593764ca-f929-4458-b700-197318ffc50b None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:10.636663 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593764ca-f929-4458-b700-197318ffc50b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:10.637097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593764ca-f929-4458-b700-197318ffc50b None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:10.637984 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593764ca-f929-4458-b700-197318ffc50b None admin] RBAC: Adding query filter params (name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:10.638455 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593764ca-f929-4458-b700-197318ffc50b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:10.640090 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593764ca-f929-4458-b700-197318ffc50b None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:10.667134 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 481/947] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Sep 12 14:01:10 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:10.685046 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:10.746049 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b9648b-2522-44d3-a455-3cdfa39f8605 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:10.784495 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b0fc4e-7089-4b3a-b473-55cd7b74d751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:10.785116 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b0fc4e-7089-4b3a-b473-55cd7b74d751 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:10.785389 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b0fc4e-7089-4b3a-b473-55cd7b74d751 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:10.785654 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b0fc4e-7089-4b3a-b473-55cd7b74d751 None admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:10.786352 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b0fc4e-7089-4b3a-b473-55cd7b74d751 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:10.786650 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b0fc4e-7089-4b3a-b473-55cd7b74d751 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:10.788734 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b0fc4e-7089-4b3a-b473-55cd7b74d751 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:10.807153 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 467/948] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 12 14:01:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:01:10.809249 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593764ca-f929-4458-b700-197318ffc50b None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:10.927638 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593764ca-f929-4458-b700-197318ffc50b None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:10.965479 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a15d90d-a5d7-4207-af0f-a951efa0a3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:10.966467 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15d90d-a5d7-4207-af0f-a951efa0a3c6 None admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:10.966947 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15d90d-a5d7-4207-af0f-a951efa0a3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:10.967406 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15d90d-a5d7-4207-af0f-a951efa0a3c6 None admin] PATH_INFO: `/v3/projects/f666a617f8d14eb18bbeb076032b82f2/users/1405e758a3af4e8ba659f498cd28ace1/roles/c5ed025905434cacbeed08d80fc31213` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:10.969819 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a15d90d-a5d7-4207-af0f-a951efa0a3c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=f666a617f8d14eb18bbeb076032b82f2, user_id=1405e758a3af4e8ba659f498cd28ace1, role_id=c5ed025905434cacbeed08d80fc31213)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:10.971453 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a15d90d-a5d7-4207-af0f-a951efa0a3c6 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:10.990996 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 482/949] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 12 14:01:10 2022] PUT /identity/v3/projects/f666a617f8d14eb18bbeb076032b82f2/users/1405e758a3af4e8ba659f498cd28ace1/roles/c5ed025905434cacbeed08d80fc31213 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:01:11.193166 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86fad78-758e-4a99-ae51-8e8bc0215dfc None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:11.194120 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86fad78-758e-4a99-ae51-8e8bc0215dfc None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:11.194648 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86fad78-758e-4a99-ae51-8e8bc0215dfc None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:11.196794 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 468/950] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:01:11.201998 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb869bec-e089-4195-a589-eedb30ff0ac6 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:11.202247 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb869bec-e089-4195-a589-eedb30ff0ac6 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:11.202520 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb869bec-e089-4195-a589-eedb30ff0ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:11.221583 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-bb869bec-e089-4195-a589-eedb30ff0ac6 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:11.223166 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-bb869bec-e089-4195-a589-eedb30ff0ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:11.244978 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:11.294762 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 483/951] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 14:01:11.562239 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86fad78-758e-4a99-ae51-8e8bc0215dfc None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:11.670052 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86fad78-758e-4a99-ae51-8e8bc0215dfc None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:11.681086 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:11.715960 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b8f995-fb2d-422b-9897-51c0706a1279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:11.716969 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8f995-fb2d-422b-9897-51c0706a1279 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:11.717459 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8f995-fb2d-422b-9897-51c0706a1279 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:11.717919 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8f995-fb2d-422b-9897-51c0706a1279 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:11.718689 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b8f995-fb2d-422b-9897-51c0706a1279 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:11.720360 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b8f995-fb2d-422b-9897-51c0706a1279 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:11.729614 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-99b8f995-fb2d-422b-9897-51c0706a1279 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 14:01:11.729614 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:11.729614 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 14:01:11.729614 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 14:01:11.729614 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 14:01:11.729614 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 14:01:11.729614 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 14:01:11.729614 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:01:11.729614 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 14:01:11.729614 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:11.729614 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 14:01:11.729614 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:11.729614 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:11.729614 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:11.729614 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:11.729614 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:11.729614 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:11.730588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:11.730588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:11.730588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:11.730588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:11.730588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:11.730588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:11.730588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 14:01:11.730588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 14:01:11.730588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 14:01:11.730588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 14:01:11.730588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:11.730588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:11.730588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:11.730588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:11.730588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:11.730588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:11.731626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:11.731626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:11.731626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:11.731626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:11.731626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:11.731626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:11.731626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:11.731626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:11.731626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:11.731626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:11.731626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 14:01:11.731626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:01:11.731626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 14:01:11.731626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:11.734373 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 469/952] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:01:11 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:11.740352 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb869bec-e089-4195-a589-eedb30ff0ac6 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:11.858607 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb869bec-e089-4195-a589-eedb30ff0ac6 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:11.867870 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442cfb3-1565-4cf9-aaa6-91d1909ef8e3 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:11.868153 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442cfb3-1565-4cf9-aaa6-91d1909ef8e3 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:11.868421 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442cfb3-1565-4cf9-aaa6-91d1909ef8e3 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:11.869969 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 470/953] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:01:11.875153 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d56b18-2a87-425a-8cf6-8690661735df None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:11.875436 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d56b18-2a87-425a-8cf6-8690661735df None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:11.875709 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d56b18-2a87-425a-8cf6-8690661735df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:11.895095 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36638a-09f4-49ef-b3bf-1e4b7494e312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:11.895796 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36638a-09f4-49ef-b3bf-1e4b7494e312 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:11.896071 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36638a-09f4-49ef-b3bf-1e4b7494e312 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:11.896341 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36638a-09f4-49ef-b3bf-1e4b7494e312 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:11.896521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-80d56b18-2a87-425a-8cf6-8690661735df None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:11.896966 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d36638a-09f4-49ef-b3bf-1e4b7494e312 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:11.897262 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d36638a-09f4-49ef-b3bf-1e4b7494e312 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:11.897602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-80d56b18-2a87-425a-8cf6-8690661735df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:11.898654 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d36638a-09f4-49ef-b3bf-1e4b7494e312 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:11.904121 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 484/954] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Sep 12 14:01:11 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:11.912210 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36638a-09f4-49ef-b3bf-1e4b7494e312 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:11.936792 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:11.989417 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 471/955] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:01:12.031245 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36638a-09f4-49ef-b3bf-1e4b7494e312 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:12.066190 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be24f8f-c1f1-4f6d-944d-190bc75656f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:12.067102 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be24f8f-c1f1-4f6d-944d-190bc75656f3 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:12.067523 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be24f8f-c1f1-4f6d-944d-190bc75656f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:12.067932 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be24f8f-c1f1-4f6d-944d-190bc75656f3 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:12.068721 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be24f8f-c1f1-4f6d-944d-190bc75656f3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:12.069219 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be24f8f-c1f1-4f6d-944d-190bc75656f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:12.071657 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be24f8f-c1f1-4f6d-944d-190bc75656f3 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:12.076794 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 485/956] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Sep 12 14:01:11 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:12.256275 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d56b18-2a87-425a-8cf6-8690661735df None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:12.365344 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d56b18-2a87-425a-8cf6-8690661735df None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:12.367473 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:12.402511 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ae7d53-158c-4131-a225-95307ec647d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:12.403173 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae7d53-158c-4131-a225-95307ec647d0 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:12.403449 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae7d53-158c-4131-a225-95307ec647d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:12.403719 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae7d53-158c-4131-a225-95307ec647d0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:12.404303 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ae7d53-158c-4131-a225-95307ec647d0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:12.405663 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ae7d53-158c-4131-a225-95307ec647d0 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:12.415408 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-40ae7d53-158c-4131-a225-95307ec647d0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 14:01:12.415408 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:12.415408 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 14:01:12.415408 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 14:01:12.415408 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 14:01:12.415408 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 14:01:12.415408 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 14:01:12.415408 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:01:12.415408 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 14:01:12.415408 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:12.415408 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 14:01:12.415408 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:12.415408 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:12.415408 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:12.415408 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:12.415408 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:12.415408 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:12.416481 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:12.416481 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:12.416481 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:12.416481 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:12.416481 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:12.416481 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:12.416481 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 14:01:12.416481 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 14:01:12.416481 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 14:01:12.416481 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 14:01:12.416481 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:12.416481 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:12.416481 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:12.416481 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:12.416481 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:12.416481 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:12.417348 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:12.417348 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:12.417348 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:12.417348 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:12.417348 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:12.417348 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:12.417348 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:12.417348 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:12.417348 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:12.417348 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:12.417348 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 14:01:12.417348 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:01:12.417348 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 14:01:12.417348 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:12.417348 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 472/957] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Sep 12 14:01:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:12.422668 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be24f8f-c1f1-4f6d-944d-190bc75656f3 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:12.512818 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:12.529087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be24f8f-c1f1-4f6d-944d-190bc75656f3 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:12.564193 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340088f4-40c7-44b7-aef0-235b35d277a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:12.565179 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340088f4-40c7-44b7-aef0-235b35d277a0 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:12.565740 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340088f4-40c7-44b7-aef0-235b35d277a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:12.566227 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340088f4-40c7-44b7-aef0-235b35d277a0 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:12.567085 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340088f4-40c7-44b7-aef0-235b35d277a0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:12.567578 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340088f4-40c7-44b7-aef0-235b35d277a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:12.569098 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340088f4-40c7-44b7-aef0-235b35d277a0 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:12.574699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 486/958] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Sep 12 14:01:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:12.580043 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ae7d53-158c-4131-a225-95307ec647d0 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:12.688423 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ae7d53-158c-4131-a225-95307ec647d0 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:12.723865 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdcfc88-9857-4275-a9ef-b2e29f971fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:12.724844 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcfc88-9857-4275-a9ef-b2e29f971fd2 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:12.725963 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcfc88-9857-4275-a9ef-b2e29f971fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:12.726428 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcfc88-9857-4275-a9ef-b2e29f971fd2 None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:12.727278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdcfc88-9857-4275-a9ef-b2e29f971fd2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:12.727772 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdcfc88-9857-4275-a9ef-b2e29f971fd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:12.729322 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdcfc88-9857-4275-a9ef-b2e29f971fd2 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:12.735016 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 473/959] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Sep 12 14:01:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:12.741060 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340088f4-40c7-44b7-aef0-235b35d277a0 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:12.853329 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340088f4-40c7-44b7-aef0-235b35d277a0 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:12.888746 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9bb9d-5875-4b73-8596-1e008762b03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:12.889441 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9bb9d-5875-4b73-8596-1e008762b03e None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:12.889621 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9bb9d-5875-4b73-8596-1e008762b03e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:12.889943 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9bb9d-5875-4b73-8596-1e008762b03e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:12.890645 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb9bb9d-5875-4b73-8596-1e008762b03e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:12.892012 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb9bb9d-5875-4b73-8596-1e008762b03e None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:12.919381 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 487/960] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:01:12 2022] POST /identity/v3/endpoints => generated 349 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:01:13.070011 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a69c45-c9e9-4845-8bb5-eec9876d8687 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:13.070496 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a69c45-c9e9-4845-8bb5-eec9876d8687 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:13.070896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a69c45-c9e9-4845-8bb5-eec9876d8687 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:13.073066 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 474/961] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:01:13.078982 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd9621-8325-421a-8c47-482dcaf15dff None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:13.079303 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd9621-8325-421a-8c47-482dcaf15dff None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:13.079532 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd9621-8325-421a-8c47-482dcaf15dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:13.087406 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:13.099490 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-58bd9621-8325-421a-8c47-482dcaf15dff None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:13.102313 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-58bd9621-8325-421a-8c47-482dcaf15dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:13.187903 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 488/962] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:01:13.274065 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9128cda-1927-46ab-a7dd-0651b85c8667 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:13.274696 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9128cda-1927-46ab-a7dd-0651b85c8667 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:13.274696 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9128cda-1927-46ab-a7dd-0651b85c8667 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:13.276254 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 475/963] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:01:13.281306 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad9fd6-5473-45b8-8a0e-ce9216dc879a None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:13.281591 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad9fd6-5473-45b8-8a0e-ce9216dc879a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:13.281867 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad9fd6-5473-45b8-8a0e-ce9216dc879a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:13.300655 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-6fad9fd6-5473-45b8-8a0e-ce9216dc879a None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:13.301842 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-6fad9fd6-5473-45b8-8a0e-ce9216dc879a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:13.370455 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 489/964] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:01:13.472061 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9128cda-1927-46ab-a7dd-0651b85c8667 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:13.505350 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:13.580223 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9128cda-1927-46ab-a7dd-0651b85c8667 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:13.617547 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b6f30f-f4f9-44df-bacf-22c040f732d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:13.618141 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6f30f-f4f9-44df-bacf-22c040f732d1 None admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:13.618414 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6f30f-f4f9-44df-bacf-22c040f732d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:13.618679 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6f30f-f4f9-44df-bacf-22c040f732d1 None admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:13.619333 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b6f30f-f4f9-44df-bacf-22c040f732d1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:13.620801 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b6f30f-f4f9-44df-bacf-22c040f732d1 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:13.623332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: WARNING keystone.server.flask.common [None req-50b6f30f-f4f9-44df-bacf-22c040f732d1 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 12 14:01:13.637845 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.sql.core [None req-50b6f30f-f4f9-44df-bacf-22c040f732d1 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 12 14:01:13.637845 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 12 14:01:13.637845 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [parameters: {'user_id': '6fa00e60c588437c8f75700cd0fc95d4', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 12 14:01:13.637845 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81596) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 12 14:01:13.653450 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad9fd6-5473-45b8-8a0e-ce9216dc879a None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:13.655271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-50b6f30f-f4f9-44df-bacf-22c040f732d1 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 12 14:01:13.655271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:13.655271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 12 14:01:13.655271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 12 14:01:13.655271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 12 14:01:13.655271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 12 14:01:13.655271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 12 14:01:13.655271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application result = self._query(query) Sep 12 14:01:13.655271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 12 14:01:13.655271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application conn.query(q) Sep 12 14:01:13.655271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 12 14:01:13.655271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 12 14:01:13.655271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 12 14:01:13.655271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application result.read() Sep 12 14:01:13.655271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 12 14:01:13.656371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 12 14:01:13.656371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 12 14:01:13.656371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 12 14:01:13.656371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 12 14:01:13.656371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 12 14:01:13.656371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 12 14:01:13.656371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 12 14:01:13.656371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 12 14:01:13.656371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:13.656371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 12 14:01:13.656371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:13.656371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:13.656371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 12 14:01:13.656371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 12 14:01:13.656371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 12 14:01:13.656371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 12 14:01:13.656371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 12 14:01:13.656371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application next(self.gen) Sep 12 14:01:13.657935 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 12 14:01:13.657935 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application yield resource Sep 12 14:01:13.657935 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 12 14:01:13.657935 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application next(self.gen) Sep 12 14:01:13.657935 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 12 14:01:13.657935 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application self.session.rollback() Sep 12 14:01:13.657935 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 12 14:01:13.657935 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application self.force_reraise() Sep 12 14:01:13.657935 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 12 14:01:13.657935 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise self.value Sep 12 14:01:13.657935 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 12 14:01:13.657935 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 12 14:01:13.657935 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 12 14:01:13.657935 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application session.commit() Sep 12 14:01:13.657935 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 12 14:01:13.657935 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 12 14:01:13.657935 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 12 14:01:13.661778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application self._prepare_impl() Sep 12 14:01:13.661778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 12 14:01:13.661778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application self.session.flush() Sep 12 14:01:13.661778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 12 14:01:13.661778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application self._flush(objects) Sep 12 14:01:13.661778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 12 14:01:13.661778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 12 14:01:13.661778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 12 14:01:13.661778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application compat.raise_( Sep 12 14:01:13.661778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 12 14:01:13.661778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception Sep 12 14:01:13.661778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 12 14:01:13.661778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application flush_context.execute() Sep 12 14:01:13.661778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 12 14:01:13.661778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rec.execute(self) Sep 12 14:01:13.661778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 12 14:01:13.661778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 12 14:01:13.663019 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 12 14:01:13.663019 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 12 14:01:13.663019 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 12 14:01:13.663019 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 12 14:01:13.663019 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 12 14:01:13.663019 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 12 14:01:13.663019 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 12 14:01:13.663019 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 12 14:01:13.663019 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 12 14:01:13.663019 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 12 14:01:13.663019 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 12 14:01:13.663019 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 12 14:01:13.663019 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 12 14:01:13.663019 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 12 14:01:13.663019 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 12 14:01:13.663019 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception Sep 12 14:01:13.664570 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 12 14:01:13.664570 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 12 14:01:13.664570 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 12 14:01:13.664570 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 12 14:01:13.664570 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 12 14:01:13.664570 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application result = self._query(query) Sep 12 14:01:13.664570 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 12 14:01:13.664570 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application conn.query(q) Sep 12 14:01:13.664570 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 12 14:01:13.664570 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 12 14:01:13.664570 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 12 14:01:13.664570 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application result.read() Sep 12 14:01:13.664570 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 12 14:01:13.664570 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 12 14:01:13.664570 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 12 14:01:13.664570 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 12 14:01:13.665594 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 12 14:01:13.665594 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 12 14:01:13.665594 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 12 14:01:13.665594 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 12 14:01:13.665594 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 12 14:01:13.665594 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 12 14:01:13.665594 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [parameters: {'user_id': '6fa00e60c588437c8f75700cd0fc95d4', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 12 14:01:13.665594 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 12 14:01:13.665594 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:13.665594 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 14:01:13.665594 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:13.665594 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:13.665594 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:13.665594 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:13.665594 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:13.668099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:13.668099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:13.668099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:13.668099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:13.668099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:13.668099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:13.668099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:13.668099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 12 14:01:13.668099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 12 14:01:13.668099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:13.668099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:13.668099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:01:13.668099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:01:13.668099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:01:13.668099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:01:13.668099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 12 14:01:13.668099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 12 14:01:13.669001 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 12 14:01:13.669001 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 12 14:01:13.669001 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 12 14:01:13.669001 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 12 14:01:13.669001 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 12 14:01:13.669001 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:13.669001 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 476/965] 10.222.0.42 () {60 vars in 1131 bytes} [Mon Sep 12 14:01:13 2022] POST /identity/v3/users => generated 160 bytes in 185 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:13.669001 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b6f30f-f4f9-44df-bacf-22c040f732d1 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:13.742762 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:13.791021 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad9fd6-5473-45b8-8a0e-ce9216dc879a None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:13.792254 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:13.805187 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b6f30f-f4f9-44df-bacf-22c040f732d1 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:13.833627 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6bcba8-34d2-4f01-832e-24e559b781d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:13.834261 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6bcba8-34d2-4f01-832e-24e559b781d2 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:13.834594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6bcba8-34d2-4f01-832e-24e559b781d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:13.835563 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6bcba8-34d2-4f01-832e-24e559b781d2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:13.847888 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6bcba8-34d2-4f01-832e-24e559b781d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:01:13.849555 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6a8802-b66a-4629-8f8c-ecf5c1822741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:13.850482 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a8802-b66a-4629-8f8c-ecf5c1822741 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:13.853534 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-6f6bcba8-34d2-4f01-832e-24e559b781d2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 14:01:13.853534 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:13.853534 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:13.853534 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:13.853534 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:13.853534 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:13.853534 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:13.853534 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:13.853534 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:13.853534 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:13.853534 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:13.853534 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:13.853534 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:01:13.853534 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:01:13.853534 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:01:13.853534 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:01:13.854457 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:01:13.854457 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:01:13.854457 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:01:13.854457 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:01:13.854457 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:13.854457 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:13.854457 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:13.854457 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:13.854457 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:13.854457 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:13.854457 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:13.854457 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:13.854457 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:13.854457 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:13.854457 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:13.854457 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:13.859599 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:13.859599 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:13.859599 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:13.859599 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:13.859599 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:01:13.859599 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:01:13.859599 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:01:13.859599 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:01:13.859599 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:01:13.859599 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:01:13.859599 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 14:01:13.859599 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:13.860320 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a8802-b66a-4629-8f8c-ecf5c1822741 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:13.860320 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a8802-b66a-4629-8f8c-ecf5c1822741 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:13.860491 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 490/966] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Sep 12 14:01:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:01:13.863388 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6bcba8-34d2-4f01-832e-24e559b781d2 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:13.869421 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6a8802-b66a-4629-8f8c-ecf5c1822741 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:13.871354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6a8802-b66a-4629-8f8c-ecf5c1822741 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:13.884357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-ba6a8802-b66a-4629-8f8c-ecf5c1822741 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 14:01:13.884357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:13.884357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:13.884357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:13.884357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:13.884357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:13.884357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:13.884357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:13.884357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:13.884357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:13.884357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:13.884357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:13.884357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:01:13.884357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:01:13.884357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:01:13.884357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:01:13.885410 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:13.885410 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:13.885410 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:01:13.885410 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:01:13.885410 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:01:13.885410 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:01:13.885410 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:13.885410 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:13.885410 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:13.885410 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:13.885410 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:13.885410 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:13.885410 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:13.885410 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:13.885410 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:13.885410 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:13.885410 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:13.885410 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:13.886358 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:13.886358 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:13.886358 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:01:13.886358 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:01:13.886358 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:01:13.886358 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:01:13.886358 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:01:13.886358 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:01:13.886358 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 14:01:13.886358 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:13.894138 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 477/967] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 14:01:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:01:13.895542 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6a8802-b66a-4629-8f8c-ecf5c1822741 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:13.983154 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6bcba8-34d2-4f01-832e-24e559b781d2 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:14.005286 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:14.005600 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6a8802-b66a-4629-8f8c-ecf5c1822741 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:14.023337 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7dd3c-efc2-48cc-9692-e70bf777c992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:14.024219 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7dd3c-efc2-48cc-9692-e70bf777c992 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:14.024357 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7dd3c-efc2-48cc-9692-e70bf777c992 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:14.024695 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7dd3c-efc2-48cc-9692-e70bf777c992 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:14.025489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc7dd3c-efc2-48cc-9692-e70bf777c992 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:14.025862 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc7dd3c-efc2-48cc-9692-e70bf777c992 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:14.027435 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc7dd3c-efc2-48cc-9692-e70bf777c992 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:14.039001 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 491/968] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Sep 12 14:01:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:14.043584 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7dd3c-efc2-48cc-9692-e70bf777c992 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:14.052775 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee63bc7-6130-4c7f-85a2-2a14f67ad796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:14.054306 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee63bc7-6130-4c7f-85a2-2a14f67ad796 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:14.054875 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee63bc7-6130-4c7f-85a2-2a14f67ad796 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:14.055415 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee63bc7-6130-4c7f-85a2-2a14f67ad796 None admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:14.056384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee63bc7-6130-4c7f-85a2-2a14f67ad796 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:14.056944 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee63bc7-6130-4c7f-85a2-2a14f67ad796 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:14.058709 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee63bc7-6130-4c7f-85a2-2a14f67ad796 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:14.099515 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 478/969] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 12 14:01:13 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:14.156333 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7dd3c-efc2-48cc-9692-e70bf777c992 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:14.195622 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c9186-e080-49ac-9a50-548ede194119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:14.196274 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c9186-e080-49ac-9a50-548ede194119 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:14.196811 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c9186-e080-49ac-9a50-548ede194119 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:14.197082 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c9186-e080-49ac-9a50-548ede194119 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:14.210491 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992c9186-e080-49ac-9a50-548ede194119 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:14.212087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992c9186-e080-49ac-9a50-548ede194119 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:14.229656 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-992c9186-e080-49ac-9a50-548ede194119 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:01:14.229656 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:14.229656 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:14.229656 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:14.229656 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:14.229656 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:14.229656 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:14.229656 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:14.229656 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:14.229656 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:14.229656 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:14.229656 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:14.229656 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:01:14.229656 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:01:14.229656 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:01:14.229656 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:01:14.230929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:14.230929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:14.230929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:14.230929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:14.230929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:14.230929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:14.230929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:14.230929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:14.230929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:14.230929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:14.230929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:14.230929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:14.230929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:14.230929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:14.230929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:14.230929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:14.230929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:01:14.232015 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:01:14.232015 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:01:14.232015 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:14.232015 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 492/970] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:01:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:14.238432 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee63bc7-6130-4c7f-85a2-2a14f67ad796 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:14.252726 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:14.336245 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069d511-37ac-4f5b-9ddb-2235f48e1dda None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:14.336593 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069d511-37ac-4f5b-9ddb-2235f48e1dda None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:14.336794 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069d511-37ac-4f5b-9ddb-2235f48e1dda None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:14.339839 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 493/971] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:01:14.343788 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe545de-31f3-4251-9199-7ede46e3bde1 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:14.344068 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe545de-31f3-4251-9199-7ede46e3bde1 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:14.344343 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe545de-31f3-4251-9199-7ede46e3bde1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:14.353462 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee63bc7-6130-4c7f-85a2-2a14f67ad796 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:14.365065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-abe545de-31f3-4251-9199-7ede46e3bde1 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:14.366595 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-abe545de-31f3-4251-9199-7ede46e3bde1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:14.390812 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426094b3-f3da-4c4b-bc48-fcf80077ae09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:14.391810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426094b3-f3da-4c4b-bc48-fcf80077ae09 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:14.392281 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426094b3-f3da-4c4b-bc48-fcf80077ae09 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:14.392726 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426094b3-f3da-4c4b-bc48-fcf80077ae09 None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:14.393558 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426094b3-f3da-4c4b-bc48-fcf80077ae09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:14.394037 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426094b3-f3da-4c4b-bc48-fcf80077ae09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:14.395591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426094b3-f3da-4c4b-bc48-fcf80077ae09 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:14.399264 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:14.413807 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 479/972] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:01:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:14.420385 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426094b3-f3da-4c4b-bc48-fcf80077ae09 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:14.450191 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 494/973] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:01:14.500798 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:14.533142 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426094b3-f3da-4c4b-bc48-fcf80077ae09 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:14.565760 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8168ed88-36da-4a4a-9c50-1b53c1a03077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:14.566384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8168ed88-36da-4a4a-9c50-1b53c1a03077 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:14.566657 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8168ed88-36da-4a4a-9c50-1b53c1a03077 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:14.566945 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8168ed88-36da-4a4a-9c50-1b53c1a03077 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:14.574521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8168ed88-36da-4a4a-9c50-1b53c1a03077 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:14.575968 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8168ed88-36da-4a4a-9c50-1b53c1a03077 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:14.587452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-8168ed88-36da-4a4a-9c50-1b53c1a03077 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 14:01:14.587452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:14.587452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:14.587452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:14.587452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:14.587452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:14.587452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:14.587452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:14.587452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:14.587452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:14.587452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:14.587452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:14.587452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:01:14.587452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:01:14.587452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:01:14.587452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:01:14.588407 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:14.588407 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:14.588407 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:01:14.588407 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:01:14.588407 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:01:14.588407 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:01:14.588407 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:14.588407 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:14.588407 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:14.588407 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:14.588407 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:14.588407 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:14.588407 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:14.588407 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:14.588407 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:14.588407 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:14.588407 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:14.588407 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:14.594495 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:14.594495 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:14.594495 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:01:14.594495 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:01:14.594495 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:01:14.594495 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:01:14.594495 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:01:14.594495 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:01:14.594495 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 14:01:14.594495 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:14.594495 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 480/974] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Sep 12 14:01:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:14.596427 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe545de-31f3-4251-9199-7ede46e3bde1 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:14.696242 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe545de-31f3-4251-9199-7ede46e3bde1 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:14.724017 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8168ed88-36da-4a4a-9c50-1b53c1a03077 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:14.730963 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff43712-deb0-4640-94e8-7e5b044af7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:14.731576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff43712-deb0-4640-94e8-7e5b044af7ec None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:14.731854 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff43712-deb0-4640-94e8-7e5b044af7ec None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:14.732123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff43712-deb0-4640-94e8-7e5b044af7ec None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:14.733271 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff43712-deb0-4640-94e8-7e5b044af7ec None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:14.733588 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff43712-deb0-4640-94e8-7e5b044af7ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:14.735377 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff43712-deb0-4640-94e8-7e5b044af7ec None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:14.754815 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 495/975] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Sep 12 14:01:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:14.760533 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff43712-deb0-4640-94e8-7e5b044af7ec None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:14.788732 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:14.836699 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8168ed88-36da-4a4a-9c50-1b53c1a03077 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:14.866211 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:14.873494 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2fdbdf-9988-452a-af4c-9b9628206e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:14.874101 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fdbdf-9988-452a-af4c-9b9628206e4c None admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:14.874365 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fdbdf-9988-452a-af4c-9b9628206e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:14.874634 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fdbdf-9988-452a-af4c-9b9628206e4c None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:14.875325 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2fdbdf-9988-452a-af4c-9b9628206e4c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:14.876675 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2fdbdf-9988-452a-af4c-9b9628206e4c None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:14.876971 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff43712-deb0-4640-94e8-7e5b044af7ec None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:14.893389 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 481/976] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Sep 12 14:01:14 2022] POST /identity/v3/services => generated 242 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:01:14.922139 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad43b8-794a-4308-96b3-e3d55ba3a753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:14.922769 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad43b8-794a-4308-96b3-e3d55ba3a753 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:14.923083 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad43b8-794a-4308-96b3-e3d55ba3a753 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:14.923355 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad43b8-794a-4308-96b3-e3d55ba3a753 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:14.999358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adad43b8-794a-4308-96b3-e3d55ba3a753 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:15.000775 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adad43b8-794a-4308-96b3-e3d55ba3a753 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:15.020899 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:15.112636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 496/977] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:01:14 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 12 14:01:15.118667 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2fdbdf-9988-452a-af4c-9b9628206e4c None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:15.189945 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20853b0-0568-4cef-adb7-a8089f2b452d None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:15.190292 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20853b0-0568-4cef-adb7-a8089f2b452d None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:15.190504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20853b0-0568-4cef-adb7-a8089f2b452d None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:15.191889 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 497/978] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:01:15.197730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475c697-8a26-4f23-ae52-1a09bf8de320 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:15.198009 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475c697-8a26-4f23-ae52-1a09bf8de320 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:15.198284 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475c697-8a26-4f23-ae52-1a09bf8de320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:15.218294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-8475c697-8a26-4f23-ae52-1a09bf8de320 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:15.219522 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-8475c697-8a26-4f23-ae52-1a09bf8de320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:15.220685 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2fdbdf-9988-452a-af4c-9b9628206e4c None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:15.255941 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7850c629-0f48-4b05-887b-409776acb697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:15.256494 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7850c629-0f48-4b05-887b-409776acb697 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:15.257312 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7850c629-0f48-4b05-887b-409776acb697 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:15.257586 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7850c629-0f48-4b05-887b-409776acb697 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:15.268158 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7850c629-0f48-4b05-887b-409776acb697 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:15.269592 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7850c629-0f48-4b05-887b-409776acb697 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:15.283942 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-7850c629-0f48-4b05-887b-409776acb697 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:01:15.283942 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:15.283942 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:15.283942 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:15.283942 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:15.283942 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:15.283942 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:15.283942 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:15.283942 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:15.283942 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:15.283942 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:15.283942 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:15.283942 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:01:15.283942 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:01:15.283942 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:01:15.283942 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:01:15.285166 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:15.285166 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:15.285166 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:15.285166 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:15.285166 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:15.285166 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:15.285166 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:15.285166 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:15.285166 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:15.285166 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:15.285166 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:15.285166 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:15.285166 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:15.285166 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:15.285166 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:15.285166 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:15.285166 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:01:15.286287 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:01:15.286287 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:01:15.286287 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:15.289560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 482/979] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:01:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:15.290858 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7850c629-0f48-4b05-887b-409776acb697 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:15.298985 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 498/980] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:15 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 14:01:15.334734 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:15.393328 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7850c629-0f48-4b05-887b-409776acb697 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:15.429332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e380966-3e87-4974-986d-1b8b690a43fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:15.429919 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e380966-3e87-4974-986d-1b8b690a43fe None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:15.430188 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e380966-3e87-4974-986d-1b8b690a43fe None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:15.430449 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e380966-3e87-4974-986d-1b8b690a43fe None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:15.431101 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e380966-3e87-4974-986d-1b8b690a43fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:15.431395 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e380966-3e87-4974-986d-1b8b690a43fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:15.432734 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e380966-3e87-4974-986d-1b8b690a43fe None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:15.444626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 483/981] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:01:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:01:15.450221 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8475c697-8a26-4f23-ae52-1a09bf8de320 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:15.500967 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:15.552510 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8475c697-8a26-4f23-ae52-1a09bf8de320 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:15.562501 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e380966-3e87-4974-986d-1b8b690a43fe None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:15.588312 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc550b-ab7e-4d39-85da-d1d83880e6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:15.588941 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc550b-ab7e-4d39-85da-d1d83880e6b6 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:15.589220 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc550b-ab7e-4d39-85da-d1d83880e6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:15.589487 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc550b-ab7e-4d39-85da-d1d83880e6b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:15.596333 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc550b-ab7e-4d39-85da-d1d83880e6b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:15.597753 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc550b-ab7e-4d39-85da-d1d83880e6b6 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:15.609654 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-0abc550b-ab7e-4d39-85da-d1d83880e6b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:01:15.609654 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:15.609654 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:15.609654 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:15.609654 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:15.609654 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:15.609654 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:15.609654 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:15.609654 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:15.609654 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:15.609654 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:15.609654 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:15.609654 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:01:15.609654 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:01:15.609654 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:01:15.610565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:01:15.610565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:15.610565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:15.610565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:15.610565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:15.610565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:15.610565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:15.610565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:15.610565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:15.610565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:15.610565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:15.610565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:15.610565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:15.610565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:15.610565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:15.610565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:15.610565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:15.612000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:01:15.612000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:01:15.612000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:01:15.612000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:01:15.612000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:01:15.612000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:01:15.612000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:01:15.612000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:15.612000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 499/982] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:01:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:15.617126 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc550b-ab7e-4d39-85da-d1d83880e6b6 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:15.673642 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e380966-3e87-4974-986d-1b8b690a43fe None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:15.675349 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:15.715632 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b2ba9-309c-481b-8e12-8e07364f67b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:15.716410 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b2ba9-309c-481b-8e12-8e07364f67b4 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:15.716702 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b2ba9-309c-481b-8e12-8e07364f67b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:15.716981 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b2ba9-309c-481b-8e12-8e07364f67b4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:15.718125 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5b2ba9-309c-481b-8e12-8e07364f67b4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:15.719699 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5b2ba9-309c-481b-8e12-8e07364f67b4 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:15.722577 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc550b-ab7e-4d39-85da-d1d83880e6b6 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:15.732109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-0d5b2ba9-309c-481b-8e12-8e07364f67b4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 14:01:15.732109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:15.732109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 14:01:15.732109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 14:01:15.732109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 14:01:15.732109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 14:01:15.732109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 14:01:15.732109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:01:15.732109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 14:01:15.732109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:15.732109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 14:01:15.732109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:15.732109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:15.732109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:15.732109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:15.732109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:15.732109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:15.733113 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:15.733113 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:15.733113 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:15.733113 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:15.733113 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:15.733113 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:15.733113 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 14:01:15.733113 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 14:01:15.733113 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 14:01:15.733113 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 14:01:15.733113 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:15.733113 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:15.733113 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:15.733113 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:15.733113 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:15.733113 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:15.733994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:15.733994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:15.733994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:15.733994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:15.733994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:15.733994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:15.733994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:15.733994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:15.733994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:15.733994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:15.733994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 14:01:15.733994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:01:15.733994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 14:01:15.733994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:15.735553 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:15.738062 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 484/983] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Sep 12 14:01:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:01:15.739074 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b2ba9-309c-481b-8e12-8e07364f67b4 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:15.763154 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475aa7c0-1043-480e-a2da-bd4c379928b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:15.763763 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475aa7c0-1043-480e-a2da-bd4c379928b9 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:15.764035 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475aa7c0-1043-480e-a2da-bd4c379928b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:15.764302 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475aa7c0-1043-480e-a2da-bd4c379928b9 None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:15.764997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475aa7c0-1043-480e-a2da-bd4c379928b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:15.765299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475aa7c0-1043-480e-a2da-bd4c379928b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:15.766710 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475aa7c0-1043-480e-a2da-bd4c379928b9 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:15.779564 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 500/984] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 12 14:01:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:01:15.785289 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475aa7c0-1043-480e-a2da-bd4c379928b9 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:15.857209 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b2ba9-309c-481b-8e12-8e07364f67b4 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:15.891828 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e9827-8c24-439a-bea6-eadb8e0280bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:15.892432 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e9827-8c24-439a-bea6-eadb8e0280bd None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:15.892711 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e9827-8c24-439a-bea6-eadb8e0280bd None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:15.892982 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e9827-8c24-439a-bea6-eadb8e0280bd None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:15.893603 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3e9827-8c24-439a-bea6-eadb8e0280bd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:15.893902 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3e9827-8c24-439a-bea6-eadb8e0280bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:15.895722 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3e9827-8c24-439a-bea6-eadb8e0280bd None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:15.901577 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 485/985] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Sep 12 14:01:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:15.907053 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e9827-8c24-439a-bea6-eadb8e0280bd None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:15.909684 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475aa7c0-1043-480e-a2da-bd4c379928b9 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:15.938098 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:15.947860 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f102c0-78fc-4974-b381-f7edf08776ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:15.948471 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f102c0-78fc-4974-b381-f7edf08776ed None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:15.948751 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f102c0-78fc-4974-b381-f7edf08776ed None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:15.949027 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f102c0-78fc-4974-b381-f7edf08776ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:15.949755 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f102c0-78fc-4974-b381-f7edf08776ed None admin] RBAC: Adding query filter params (role.id=c5ed025905434cacbeed08d80fc31213, scope.project.id=f666a617f8d14eb18bbeb076032b82f2, user.id=1405e758a3af4e8ba659f498cd28ace1) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:15.950063 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f102c0-78fc-4974-b381-f7edf08776ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:15.951542 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f102c0-78fc-4974-b381-f7edf08776ed None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:15.966863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 501/986] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 14:01:15 2022] GET /identity/v3/role_assignments?user.id=1405e758a3af4e8ba659f498cd28ace1&scope.project.id=f666a617f8d14eb18bbeb076032b82f2&role.id=c5ed025905434cacbeed08d80fc31213 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:01:16.011973 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e9827-8c24-439a-bea6-eadb8e0280bd None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:16.048553 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ccbce7-722e-4853-938f-69b9c3e469c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:16.049167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ccbce7-722e-4853-938f-69b9c3e469c3 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:16.049440 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ccbce7-722e-4853-938f-69b9c3e469c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:16.049706 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ccbce7-722e-4853-938f-69b9c3e469c3 None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:16.050338 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ccbce7-722e-4853-938f-69b9c3e469c3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:16.050634 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ccbce7-722e-4853-938f-69b9c3e469c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:16.052068 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ccbce7-722e-4853-938f-69b9c3e469c3 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:16.057455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 486/987] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Sep 12 14:01:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:01:16.063629 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f102c0-78fc-4974-b381-f7edf08776ed None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:16.098764 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:16.174641 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f102c0-78fc-4974-b381-f7edf08776ed None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:16.211763 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95c6caf-4fa1-47e2-924c-5cfea1f193fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:16.212373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c6caf-4fa1-47e2-924c-5cfea1f193fe None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:16.212646 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c6caf-4fa1-47e2-924c-5cfea1f193fe None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:16.212912 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c6caf-4fa1-47e2-924c-5cfea1f193fe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:16.213612 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95c6caf-4fa1-47e2-924c-5cfea1f193fe None admin] RBAC: Adding query filter params (interface=admin, service_id=bd4911ae3aba42028c13a9ec4c66e893, region_id=RegionOne) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:16.213913 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95c6caf-4fa1-47e2-924c-5cfea1f193fe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:16.215287 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95c6caf-4fa1-47e2-924c-5cfea1f193fe None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:16.220547 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 502/988] 10.222.0.42 () {56 vars in 1249 bytes} [Mon Sep 12 14:01:16 2022] GET /identity/v3/endpoints?service_id=bd4911ae3aba42028c13a9ec4c66e893&interface=admin®ion_id=RegionOne => generated 193 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:16.226387 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ccbce7-722e-4853-938f-69b9c3e469c3 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:16.320890 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:16.329020 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ccbce7-722e-4853-938f-69b9c3e469c3 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:16.362489 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde7b9b-721e-4895-baa2-32f282dd8392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:16.363176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde7b9b-721e-4895-baa2-32f282dd8392 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:16.363546 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde7b9b-721e-4895-baa2-32f282dd8392 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:16.363901 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde7b9b-721e-4895-baa2-32f282dd8392 None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:16.364587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbde7b9b-721e-4895-baa2-32f282dd8392 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:16.366081 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbde7b9b-721e-4895-baa2-32f282dd8392 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:16.378882 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 487/989] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Sep 12 14:01:16 2022] GET /identity/v3/services => generated 1727 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:01:17.117726 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c237f-e0ae-4de7-a798-63d198945910 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:17.118219 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c237f-e0ae-4de7-a798-63d198945910 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:17.118219 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c237f-e0ae-4de7-a798-63d198945910 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:17.119707 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 503/990] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:01:17.124692 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50a6e6-6161-4205-b0a5-6e70845b3dbf None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:17.125108 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50a6e6-6161-4205-b0a5-6e70845b3dbf None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:17.125289 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50a6e6-6161-4205-b0a5-6e70845b3dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:17.143322 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-5c50a6e6-6161-4205-b0a5-6e70845b3dbf None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:17.144808 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-5c50a6e6-6161-4205-b0a5-6e70845b3dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:17.214573 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 488/991] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:17 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:01:17.486141 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c237f-e0ae-4de7-a798-63d198945910 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:17.585110 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:17.593168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c237f-e0ae-4de7-a798-63d198945910 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:17.626628 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef733a-f2bc-46d7-927c-19619c72d53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:17.627272 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef733a-f2bc-46d7-927c-19619c72d53a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:17.627548 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef733a-f2bc-46d7-927c-19619c72d53a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:17.627823 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef733a-f2bc-46d7-927c-19619c72d53a None admin] PATH_INFO: `/v3/services/compute` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:17.628400 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ef733a-f2bc-46d7-927c-19619c72d53a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:17.629736 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ef733a-f2bc-46d7-927c-19619c72d53a None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:17.638661 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-f8ef733a-f2bc-46d7-927c-19619c72d53a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 14:01:17.638661 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:17.638661 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 14:01:17.638661 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 14:01:17.638661 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 14:01:17.638661 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 14:01:17.638661 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 14:01:17.638661 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:01:17.638661 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 14:01:17.638661 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:17.638661 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 14:01:17.638661 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:17.638661 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:17.638661 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:17.638661 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:17.638661 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:17.638661 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:17.639723 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:17.639723 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:17.639723 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:17.639723 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:17.639723 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:17.639723 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:17.639723 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 14:01:17.639723 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 14:01:17.639723 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 14:01:17.639723 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 14:01:17.639723 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:17.639723 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:17.639723 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:17.639723 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:17.639723 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:17.639723 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:17.640586 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:17.640586 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:17.640586 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:17.640586 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:17.640586 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:17.640586 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:17.640586 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:17.640586 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:17.640586 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:17.640586 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:17.640586 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 14:01:17.640586 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:01:17.640586 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 14:01:17.640586 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:17.640586 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 504/992] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:01:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:17.645531 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c50a6e6-6161-4205-b0a5-6e70845b3dbf None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:17.733748 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:17.747464 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c50a6e6-6161-4205-b0a5-6e70845b3dbf None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:17.781136 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916be4f-9291-4da3-8d16-cf6f385e0ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:17.781732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916be4f-9291-4da3-8d16-cf6f385e0ede None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:17.782005 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916be4f-9291-4da3-8d16-cf6f385e0ede None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:17.782270 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916be4f-9291-4da3-8d16-cf6f385e0ede None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:17.782909 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0916be4f-9291-4da3-8d16-cf6f385e0ede None admin] RBAC: Adding query filter params (name=compute) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:17.783207 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0916be4f-9291-4da3-8d16-cf6f385e0ede None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:17.784545 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0916be4f-9291-4da3-8d16-cf6f385e0ede None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:17.794794 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 489/993] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Sep 12 14:01:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:17.796099 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef733a-f2bc-46d7-927c-19619c72d53a None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:17.902970 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef733a-f2bc-46d7-927c-19619c72d53a None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:17.936118 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c9bb6-e662-429d-8157-b99560ad7cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:17.936716 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c9bb6-e662-429d-8157-b99560ad7cb5 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:17.936990 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c9bb6-e662-429d-8157-b99560ad7cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:17.937292 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c9bb6-e662-429d-8157-b99560ad7cb5 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:17.937940 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0c9bb6-e662-429d-8157-b99560ad7cb5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:17.938238 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0c9bb6-e662-429d-8157-b99560ad7cb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:17.939635 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0c9bb6-e662-429d-8157-b99560ad7cb5 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:17.945155 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 505/994] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Sep 12 14:01:17 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:17.950996 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916be4f-9291-4da3-8d16-cf6f385e0ede None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:18.052118 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916be4f-9291-4da3-8d16-cf6f385e0ede None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:18.090144 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ce6e7-2b13-4560-a7f7-542e0178888a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:18.090755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ce6e7-2b13-4560-a7f7-542e0178888a None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:18.091059 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ce6e7-2b13-4560-a7f7-542e0178888a None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:18.091330 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ce6e7-2b13-4560-a7f7-542e0178888a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:18.092022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ce6e7-2b13-4560-a7f7-542e0178888a None admin] RBAC: Adding query filter params (interface=public, service_id=29bf61050d724a70957c362418f82808, region_id=RegionOne) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:18.092325 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ce6e7-2b13-4560-a7f7-542e0178888a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:18.093674 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ce6e7-2b13-4560-a7f7-542e0178888a None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:18.103443 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 490/995] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Sep 12 14:01:17 2022] GET /identity/v3/endpoints?service_id=29bf61050d724a70957c362418f82808&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:18.105037 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c9bb6-e662-429d-8157-b99560ad7cb5 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:18.130364 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:18.164897 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8719b58-cc1e-48e8-a9bf-be16ab4bc20a None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:18.165437 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8719b58-cc1e-48e8-a9bf-be16ab4bc20a None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:18.165437 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8719b58-cc1e-48e8-a9bf-be16ab4bc20a None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:18.166757 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 491/996] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 14:01:18.174864 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb4fb6-5c25-4d03-9c46-eeedb98e4637 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:18.174864 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb4fb6-5c25-4d03-9c46-eeedb98e4637 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:18.174864 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb4fb6-5c25-4d03-9c46-eeedb98e4637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:18.191589 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-e1fb4fb6-5c25-4d03-9c46-eeedb98e4637 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:18.192775 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-e1fb4fb6-5c25-4d03-9c46-eeedb98e4637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:18.204481 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:18.214057 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c9bb6-e662-429d-8157-b99560ad7cb5 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:18.248510 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee65f6-d31e-453d-bded-b17e60437808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:18.257411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee65f6-d31e-453d-bded-b17e60437808 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:18.257411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee65f6-d31e-453d-bded-b17e60437808 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:18.257411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee65f6-d31e-453d-bded-b17e60437808 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:18.257411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee65f6-d31e-453d-bded-b17e60437808 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:18.257411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee65f6-d31e-453d-bded-b17e60437808 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:18.260103 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 506/997] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Sep 12 14:01:18 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:01:18.280627 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 492/998] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:18 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:01:18.551721 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee65f6-d31e-453d-bded-b17e60437808 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:18.567943 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70efb9e6-b759-4214-9032-62bda6569501 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:18.568259 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70efb9e6-b759-4214-9032-62bda6569501 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:18.568496 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70efb9e6-b759-4214-9032-62bda6569501 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:18.570071 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 493/999] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:01:18.575076 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc423e-c12b-42e1-b0e6-ef499659e652 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:18.575362 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc423e-c12b-42e1-b0e6-ef499659e652 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:18.575638 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc423e-c12b-42e1-b0e6-ef499659e652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:18.593656 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-4bdc423e-c12b-42e1-b0e6-ef499659e652 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:18.594855 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-4bdc423e-c12b-42e1-b0e6-ef499659e652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:18.648567 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:18.661655 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee65f6-d31e-453d-bded-b17e60437808 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:18.664945 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 494/1000] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:18 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 14:01:18.697378 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c821b038-0f03-4f43-8261-f6daf608ef1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:18.698011 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821b038-0f03-4f43-8261-f6daf608ef1e None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:18.698381 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821b038-0f03-4f43-8261-f6daf608ef1e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:18.698658 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821b038-0f03-4f43-8261-f6daf608ef1e None admin] PATH_INFO: `/v3/services/image` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:18.699262 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c821b038-0f03-4f43-8261-f6daf608ef1e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:18.700605 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c821b038-0f03-4f43-8261-f6daf608ef1e None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:18.709576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-c821b038-0f03-4f43-8261-f6daf608ef1e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 14:01:18.709576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:18.709576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 14:01:18.709576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 14:01:18.709576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 14:01:18.709576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 14:01:18.709576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 14:01:18.709576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:01:18.709576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 14:01:18.709576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:18.709576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 14:01:18.709576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:18.709576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:18.709576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:18.709576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:18.709576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:18.709576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:18.710629 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:18.710629 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:18.710629 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:18.710629 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:18.710629 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:18.710629 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:18.710629 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 14:01:18.710629 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 14:01:18.710629 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 14:01:18.710629 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 14:01:18.710629 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:18.710629 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:18.710629 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:18.710629 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:18.710629 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:18.710629 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:18.716080 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:18.716080 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:18.716080 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:18.716080 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:18.716080 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:18.716080 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:18.716080 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:18.716080 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:18.716080 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:18.716080 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:18.716080 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 14:01:18.716080 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:01:18.716080 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 14:01:18.716080 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:18.716080 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 507/1001] 10.222.0.42 () {56 vars in 1105 bytes} [Mon Sep 12 14:01:18 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:18.716953 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc423e-c12b-42e1-b0e6-ef499659e652 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:18.795934 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:18.823162 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc423e-c12b-42e1-b0e6-ef499659e652 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:18.873371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7afe1e-50ee-4332-9fdd-4cf71634a836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:18.873977 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7afe1e-50ee-4332-9fdd-4cf71634a836 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:18.874251 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7afe1e-50ee-4332-9fdd-4cf71634a836 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:18.874619 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7afe1e-50ee-4332-9fdd-4cf71634a836 None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:18.875320 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7afe1e-50ee-4332-9fdd-4cf71634a836 None admin] RBAC: Adding query filter params (name=image) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:18.875625 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7afe1e-50ee-4332-9fdd-4cf71634a836 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:18.877025 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7afe1e-50ee-4332-9fdd-4cf71634a836 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:18.886700 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 495/1002] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:01:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:18.887965 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c821b038-0f03-4f43-8261-f6daf608ef1e None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:18.937223 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7afe1e-50ee-4332-9fdd-4cf71634a836 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:19.002829 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c821b038-0f03-4f43-8261-f6daf608ef1e None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:19.041108 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7afe1e-50ee-4332-9fdd-4cf71634a836 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:19.041491 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e73ec88-9edc-4830-ac73-e857da4c862b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:19.042100 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e73ec88-9edc-4830-ac73-e857da4c862b None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:19.042381 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e73ec88-9edc-4830-ac73-e857da4c862b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:19.042686 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e73ec88-9edc-4830-ac73-e857da4c862b None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:19.043310 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e73ec88-9edc-4830-ac73-e857da4c862b None admin] RBAC: Adding query filter params (type=image) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:19.044227 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e73ec88-9edc-4830-ac73-e857da4c862b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:19.044970 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e73ec88-9edc-4830-ac73-e857da4c862b None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:19.049765 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 508/1003] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:01:18 2022] GET /identity/v3/services?type=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:19.079177 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bc101-db49-4d5b-bfd4-9e1c1c922db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:19.079750 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bc101-db49-4d5b-bfd4-9e1c1c922db3 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:19.080032 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bc101-db49-4d5b-bfd4-9e1c1c922db3 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:19.080306 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bc101-db49-4d5b-bfd4-9e1c1c922db3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:19.080908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119bc101-db49-4d5b-bfd4-9e1c1c922db3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:19.082304 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119bc101-db49-4d5b-bfd4-9e1c1c922db3 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:19.092810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-119bc101-db49-4d5b-bfd4-9e1c1c922db3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 14:01:19.092810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:19.092810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 14:01:19.092810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 14:01:19.092810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 14:01:19.092810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 14:01:19.092810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 14:01:19.092810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:01:19.092810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 14:01:19.092810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:19.092810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 14:01:19.092810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:19.092810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:19.092810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:19.092810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:19.092810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:19.092810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:19.093908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:19.093908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:19.093908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:19.093908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:19.093908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:19.093908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:19.093908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 14:01:19.093908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 14:01:19.093908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 14:01:19.093908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 14:01:19.093908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:19.093908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:19.093908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:19.093908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:19.093908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:19.093908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:19.094739 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:19.094739 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:19.094739 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:19.094739 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:19.094739 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:19.094739 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:19.094739 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:19.094739 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:19.094739 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:19.094739 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:19.094739 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 14:01:19.094739 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:01:19.094739 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 14:01:19.094739 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:19.094739 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 496/1004] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Sep 12 14:01:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:01:19.100135 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e73ec88-9edc-4830-ac73-e857da4c862b None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:19.123199 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:19.213724 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e73ec88-9edc-4830-ac73-e857da4c862b None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:19.249801 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43c6a93-6b69-4529-99df-983bb2660492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:19.250414 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c6a93-6b69-4529-99df-983bb2660492 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:19.250695 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c6a93-6b69-4529-99df-983bb2660492 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:19.250997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c6a93-6b69-4529-99df-983bb2660492 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:19.251625 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43c6a93-6b69-4529-99df-983bb2660492 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:19.251923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43c6a93-6b69-4529-99df-983bb2660492 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:19.253268 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43c6a93-6b69-4529-99df-983bb2660492 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:19.258719 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 509/1005] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Sep 12 14:01:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:19.264188 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bc101-db49-4d5b-bfd4-9e1c1c922db3 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:19.268117 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:19.365055 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bc101-db49-4d5b-bfd4-9e1c1c922db3 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:19.399117 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b9841a-203f-47ab-96ad-a3f9ad05cd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:19.399727 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9841a-203f-47ab-96ad-a3f9ad05cd7f None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:19.400007 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9841a-203f-47ab-96ad-a3f9ad05cd7f None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:19.400277 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9841a-203f-47ab-96ad-a3f9ad05cd7f None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:19.400902 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b9841a-203f-47ab-96ad-a3f9ad05cd7f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:19.401205 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b9841a-203f-47ab-96ad-a3f9ad05cd7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:19.402561 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b9841a-203f-47ab-96ad-a3f9ad05cd7f None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:19.407573 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 497/1006] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Sep 12 14:01:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:19.413090 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43c6a93-6b69-4529-99df-983bb2660492 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:19.495394 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:19.516021 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43c6a93-6b69-4529-99df-983bb2660492 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:19.550017 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3b44e-73d0-4222-a96b-d59d80a20211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:19.550625 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3b44e-73d0-4222-a96b-d59d80a20211 None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:19.550928 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3b44e-73d0-4222-a96b-d59d80a20211 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:19.551200 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3b44e-73d0-4222-a96b-d59d80a20211 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:19.551872 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c3b44e-73d0-4222-a96b-d59d80a20211 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:19.553216 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c3b44e-73d0-4222-a96b-d59d80a20211 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:19.574584 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 510/1007] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:01:19 2022] POST /identity/v3/endpoints => generated 325 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:01:20.428018 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f2b65-bdfb-4ae6-a342-cc3626daff9b None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:20.429075 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f2b65-bdfb-4ae6-a342-cc3626daff9b None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:20.429162 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f2b65-bdfb-4ae6-a342-cc3626daff9b None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:20.430879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 498/1008] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:01:20.435799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2d55d-1c2f-4d16-bf3c-47ad433cce7e None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:20.436209 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2d55d-1c2f-4d16-bf3c-47ad433cce7e None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:20.436381 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2d55d-1c2f-4d16-bf3c-47ad433cce7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:20.455314 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-87a2d55d-1c2f-4d16-bf3c-47ad433cce7e None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:20.456495 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-87a2d55d-1c2f-4d16-bf3c-47ad433cce7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:20.538094 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 511/1009] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:20 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:01:20.805471 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6f2b65-bdfb-4ae6-a342-cc3626daff9b None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:20.880115 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:20.916181 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6f2b65-bdfb-4ae6-a342-cc3626daff9b None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:20.949131 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe3e0e-588d-46b2-bc2e-2d60b98d9f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:20.949755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe3e0e-588d-46b2-bc2e-2d60b98d9f1a None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:20.950028 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe3e0e-588d-46b2-bc2e-2d60b98d9f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:20.950293 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe3e0e-588d-46b2-bc2e-2d60b98d9f1a None admin] PATH_INFO: `/v3/services/compute` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:20.950881 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fe3e0e-588d-46b2-bc2e-2d60b98d9f1a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:20.952238 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fe3e0e-588d-46b2-bc2e-2d60b98d9f1a None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:20.960842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-11fe3e0e-588d-46b2-bc2e-2d60b98d9f1a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 14:01:20.960842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:20.960842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 14:01:20.960842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 14:01:20.960842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 14:01:20.960842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 14:01:20.960842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 14:01:20.960842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:01:20.960842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 14:01:20.960842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:20.960842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 14:01:20.960842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:20.960842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:20.960842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:20.960842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:20.960842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:20.960842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:20.961838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:20.961838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:20.961838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:20.961838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:20.961838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:20.961838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:20.961838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 14:01:20.961838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 14:01:20.961838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 14:01:20.961838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 14:01:20.961838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:20.961838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:20.961838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:20.961838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:20.961838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:20.961838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:20.962682 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:20.962682 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:20.962682 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:20.962682 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:20.962682 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:20.962682 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:20.962682 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:20.962682 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:20.962682 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:20.962682 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:20.962682 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 14:01:20.962682 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:01:20.962682 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 14:01:20.962682 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:20.966742 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 499/1010] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:01:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:20.968091 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2d55d-1c2f-4d16-bf3c-47ad433cce7e None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:21.033545 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:21.069906 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2d55d-1c2f-4d16-bf3c-47ad433cce7e None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:21.105257 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3157ba-ab11-4be6-b246-44d1c05b7c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:21.105882 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3157ba-ab11-4be6-b246-44d1c05b7c61 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:21.106158 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3157ba-ab11-4be6-b246-44d1c05b7c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:21.106527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3157ba-ab11-4be6-b246-44d1c05b7c61 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:21.107260 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3157ba-ab11-4be6-b246-44d1c05b7c61 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:21.107558 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3157ba-ab11-4be6-b246-44d1c05b7c61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:21.108953 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3157ba-ab11-4be6-b246-44d1c05b7c61 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:21.118877 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 512/1011] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Sep 12 14:01:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:21.121073 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe3e0e-588d-46b2-bc2e-2d60b98d9f1a None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:21.218874 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec912a5-852f-4864-8eda-e398ed96155d None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:21.219193 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec912a5-852f-4864-8eda-e398ed96155d None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:21.219464 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec912a5-852f-4864-8eda-e398ed96155d None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:21.220791 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 513/1012] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:01:21.226462 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143062ff-9875-4f54-9109-5b11ee7f58a9 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:21.226742 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143062ff-9875-4f54-9109-5b11ee7f58a9 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:21.227038 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143062ff-9875-4f54-9109-5b11ee7f58a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:21.227412 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe3e0e-588d-46b2-bc2e-2d60b98d9f1a None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:21.244769 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-143062ff-9875-4f54-9109-5b11ee7f58a9 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:21.245942 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-143062ff-9875-4f54-9109-5b11ee7f58a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:21.258417 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:21.264247 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82e09c6-8019-44b5-81b2-fb2b54e50434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:21.265257 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e09c6-8019-44b5-81b2-fb2b54e50434 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:21.265539 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e09c6-8019-44b5-81b2-fb2b54e50434 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:21.265813 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e09c6-8019-44b5-81b2-fb2b54e50434 None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:21.266445 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82e09c6-8019-44b5-81b2-fb2b54e50434 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:21.266752 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82e09c6-8019-44b5-81b2-fb2b54e50434 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:21.268134 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82e09c6-8019-44b5-81b2-fb2b54e50434 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:21.278054 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 500/1013] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Sep 12 14:01:21 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:21.279004 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82e09c6-8019-44b5-81b2-fb2b54e50434 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:21.318563 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 514/1014] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:21 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 14:01:21.384690 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82e09c6-8019-44b5-81b2-fb2b54e50434 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:21.419626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd0f63c-4a9a-409d-b265-f9fe36d82c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:21.420226 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0f63c-4a9a-409d-b265-f9fe36d82c25 None admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:21.420504 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0f63c-4a9a-409d-b265-f9fe36d82c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:21.420830 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0f63c-4a9a-409d-b265-f9fe36d82c25 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:21.421497 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd0f63c-4a9a-409d-b265-f9fe36d82c25 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:21.423027 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd0f63c-4a9a-409d-b265-f9fe36d82c25 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:21.448806 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 501/1015] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:01:21 2022] POST /identity/v3/endpoints => generated 334 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:01:21.586287 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143062ff-9875-4f54-9109-5b11ee7f58a9 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:21.650956 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:21.697350 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143062ff-9875-4f54-9109-5b11ee7f58a9 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:21.742091 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95610f-8203-491f-847a-e86b4f8dd34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:21.742702 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95610f-8203-491f-847a-e86b4f8dd34f None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:21.743014 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95610f-8203-491f-847a-e86b4f8dd34f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:21.743285 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95610f-8203-491f-847a-e86b4f8dd34f None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:21.743950 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95610f-8203-491f-847a-e86b4f8dd34f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:21.745326 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95610f-8203-491f-847a-e86b4f8dd34f None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:21.762957 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 515/1016] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Sep 12 14:01:21 2022] POST /identity/v3/services => generated 242 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:01:21.772809 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0ee01-b945-45ab-8c15-422f3e1221db None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:21.773167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0ee01-b945-45ab-8c15-422f3e1221db None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:21.773353 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0ee01-b945-45ab-8c15-422f3e1221db None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:21.774976 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 502/1017] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:01:21.782136 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bf0f57-48cb-4d61-a548-671aed610c17 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:21.782600 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bf0f57-48cb-4d61-a548-671aed610c17 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:21.782694 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bf0f57-48cb-4d61-a548-671aed610c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:21.803174 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-f6bf0f57-48cb-4d61-a548-671aed610c17 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:21.804945 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f6bf0f57-48cb-4d61-a548-671aed610c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:21.890791 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 516/1018] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:01:22.165669 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b0ee01-b945-45ab-8c15-422f3e1221db None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:22.219575 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:22.275397 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b0ee01-b945-45ab-8c15-422f3e1221db None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:22.309686 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f64379f-30b7-44e7-b558-b107343d750c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:22.310313 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f64379f-30b7-44e7-b558-b107343d750c None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:22.310589 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f64379f-30b7-44e7-b558-b107343d750c None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:22.310876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f64379f-30b7-44e7-b558-b107343d750c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:22.311975 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f64379f-30b7-44e7-b558-b107343d750c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:22.313391 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f64379f-30b7-44e7-b558-b107343d750c None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:22.315137 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 503/1019] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:01:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:22.320729 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bf0f57-48cb-4d61-a548-671aed610c17 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:22.362943 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:22.419884 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bf0f57-48cb-4d61-a548-671aed610c17 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:22.455768 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7d1ad-8da4-407c-a4f0-35173f4c1ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:22.456381 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7d1ad-8da4-407c-a4f0-35173f4c1ae1 None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:22.456658 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7d1ad-8da4-407c-a4f0-35173f4c1ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:22.456930 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7d1ad-8da4-407c-a4f0-35173f4c1ae1 None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:22.457579 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b7d1ad-8da4-407c-a4f0-35173f4c1ae1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:22.459029 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b7d1ad-8da4-407c-a4f0-35173f4c1ae1 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:22.489443 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 517/1020] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Sep 12 14:01:22 2022] POST /identity/v3/projects => generated 311 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:01:23.720098 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78bd2b-a9df-4aa3-b7eb-ebbf9b6ba9d9 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:23.720673 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78bd2b-a9df-4aa3-b7eb-ebbf9b6ba9d9 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:23.720673 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78bd2b-a9df-4aa3-b7eb-ebbf9b6ba9d9 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:23.721989 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 504/1021] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:01:23.727388 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885987a-3069-4628-8774-bcdf02a06cb9 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:23.727790 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885987a-3069-4628-8774-bcdf02a06cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:23.727970 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885987a-3069-4628-8774-bcdf02a06cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:23.746165 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-d885987a-3069-4628-8774-bcdf02a06cb9 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:23.747372 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d885987a-3069-4628-8774-bcdf02a06cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:23.823270 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 518/1022] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:23 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:01:23.964560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81bc26-60b3-4c88-9089-a1140a11e164 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:23.964876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81bc26-60b3-4c88-9089-a1140a11e164 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:23.965114 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81bc26-60b3-4c88-9089-a1140a11e164 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:23.966590 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 505/1023] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:01:23.971630 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b41c4a-88b7-4e85-b175-2965b83f10dc None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:23.971864 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b41c4a-88b7-4e85-b175-2965b83f10dc None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:23.972133 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b41c4a-88b7-4e85-b175-2965b83f10dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:23.990210 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-a4b41c4a-88b7-4e85-b175-2965b83f10dc None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:23.991462 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-a4b41c4a-88b7-4e85-b175-2965b83f10dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:24.009730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:24.060701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 519/1024] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:23 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:01:24.097364 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a81bc26-60b3-4c88-9089-a1140a11e164 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:24.201122 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a81bc26-60b3-4c88-9089-a1140a11e164 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:24.220459 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:24.235519 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7420fefd-21a4-42f4-b589-d73bd805e234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:24.236081 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420fefd-21a4-42f4-b589-d73bd805e234 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:24.236354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420fefd-21a4-42f4-b589-d73bd805e234 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:24.236619 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420fefd-21a4-42f4-b589-d73bd805e234 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:24.247739 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7420fefd-21a4-42f4-b589-d73bd805e234 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:01:24.252524 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-7420fefd-21a4-42f4-b589-d73bd805e234 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 14:01:24.252524 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:24.252524 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:24.252524 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:24.252524 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:24.252524 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:24.252524 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:24.252524 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:24.252524 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:24.252524 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:24.252524 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:24.252524 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:24.252524 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:01:24.252524 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:01:24.252524 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:01:24.252524 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:01:24.253430 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:01:24.253430 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:01:24.253430 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:01:24.253430 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:01:24.253430 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:24.253430 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:24.253430 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:24.253430 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:24.253430 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:24.253430 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:24.253430 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:24.253430 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:24.253430 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:24.253430 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:24.253430 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:24.253430 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:24.254274 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:24.254274 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:24.254274 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:24.254274 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:24.254274 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:01:24.254274 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:01:24.254274 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:01:24.254274 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:01:24.254274 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:01:24.254274 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:01:24.254274 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 14:01:24.254274 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:24.259395 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 506/1025] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Sep 12 14:01:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:24.260386 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b41c4a-88b7-4e85-b175-2965b83f10dc None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:24.330201 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7420fefd-21a4-42f4-b589-d73bd805e234 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:24.363845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b41c4a-88b7-4e85-b175-2965b83f10dc None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:24.420071 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff629a37-2e63-4aa9-809f-8776ac03976c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:24.420704 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff629a37-2e63-4aa9-809f-8776ac03976c None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:24.421034 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff629a37-2e63-4aa9-809f-8776ac03976c None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:24.421308 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff629a37-2e63-4aa9-809f-8776ac03976c None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:24.421983 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff629a37-2e63-4aa9-809f-8776ac03976c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:24.422285 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff629a37-2e63-4aa9-809f-8776ac03976c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:24.424195 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff629a37-2e63-4aa9-809f-8776ac03976c None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:24.439019 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 520/1026] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Sep 12 14:01:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:24.442678 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff629a37-2e63-4aa9-809f-8776ac03976c None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:24.456032 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7420fefd-21a4-42f4-b589-d73bd805e234 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:24.466962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:24.495535 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0bc614-9b34-4031-bb79-e0257c201122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:24.496401 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0bc614-9b34-4031-bb79-e0257c201122 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:24.496829 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0bc614-9b34-4031-bb79-e0257c201122 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:24.497264 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0bc614-9b34-4031-bb79-e0257c201122 None admin] PATH_INFO: `/v3/services/image` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:24.498036 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0bc614-9b34-4031-bb79-e0257c201122 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:24.499594 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0bc614-9b34-4031-bb79-e0257c201122 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:24.508866 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-bf0bc614-9b34-4031-bb79-e0257c201122 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 14:01:24.508866 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:24.508866 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 14:01:24.508866 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 14:01:24.508866 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 14:01:24.508866 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 14:01:24.508866 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 14:01:24.508866 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:01:24.508866 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 14:01:24.508866 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:24.508866 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 14:01:24.508866 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:24.508866 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:24.508866 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:24.508866 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:24.508866 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:24.508866 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:24.509896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:24.509896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:24.509896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:24.509896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:24.509896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:24.509896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:24.509896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 14:01:24.509896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 14:01:24.509896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 14:01:24.509896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 14:01:24.509896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:24.509896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:24.509896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:24.509896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:24.509896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:24.509896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:24.510732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:24.510732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:24.510732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:24.510732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:24.510732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:24.510732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:24.510732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:24.510732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:24.510732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:24.510732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:24.510732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 14:01:24.510732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:01:24.510732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 14:01:24.510732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:24.513607 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 507/1027] 10.222.0.42 () {56 vars in 1105 bytes} [Mon Sep 12 14:01:24 2022] GET /identity/v3/services/image => generated 86 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:01:24.519434 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0bc614-9b34-4031-bb79-e0257c201122 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:24.556930 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff629a37-2e63-4aa9-809f-8776ac03976c None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:24.581557 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:24.598693 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db8604-65c4-4eb9-a771-6451a02e7d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:24.599481 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db8604-65c4-4eb9-a771-6451a02e7d50 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:24.599822 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db8604-65c4-4eb9-a771-6451a02e7d50 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:24.600699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db8604-65c4-4eb9-a771-6451a02e7d50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:24.612332 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db8604-65c4-4eb9-a771-6451a02e7d50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:24.614126 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db8604-65c4-4eb9-a771-6451a02e7d50 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:24.635178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-74db8604-65c4-4eb9-a771-6451a02e7d50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:01:24.635178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:24.635178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:24.635178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:24.635178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:24.635178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:24.635178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:24.635178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:24.635178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:24.635178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:24.635178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:24.635178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:24.635178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:01:24.635178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:01:24.635178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:01:24.635178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:01:24.636433 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:24.636433 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:24.636433 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:24.636433 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:24.636433 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:24.636433 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:24.636433 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:24.636433 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:24.636433 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:24.636433 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:24.636433 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:24.636433 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:24.636433 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:24.636433 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:24.636433 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:24.636433 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:24.636433 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:01:24.637468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:01:24.637468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:01:24.637468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:24.637468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 521/1028] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:01:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:24.643582 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db8604-65c4-4eb9-a771-6451a02e7d50 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:24.649858 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0bc614-9b34-4031-bb79-e0257c201122 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:24.685626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453d2286-edbb-4d8e-97cc-157ddf62c060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:24.686236 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453d2286-edbb-4d8e-97cc-157ddf62c060 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:24.686516 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453d2286-edbb-4d8e-97cc-157ddf62c060 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:24.686787 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453d2286-edbb-4d8e-97cc-157ddf62c060 None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:24.687555 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453d2286-edbb-4d8e-97cc-157ddf62c060 None admin] RBAC: Adding query filter params (name=image) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:24.687860 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453d2286-edbb-4d8e-97cc-157ddf62c060 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:24.689226 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453d2286-edbb-4d8e-97cc-157ddf62c060 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:24.694969 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 508/1029] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:01:24 2022] GET /identity/v3/services?name=image => generated 122 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:24.696176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17c42a-4b19-4b59-9344-3026ed64b293 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:24.696723 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17c42a-4b19-4b59-9344-3026ed64b293 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:24.696994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17c42a-4b19-4b59-9344-3026ed64b293 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:24.698345 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 509/1030] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:01:24.700483 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d17c42a-4b19-4b59-9344-3026ed64b293 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:24.727883 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:24.764258 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db8604-65c4-4eb9-a771-6451a02e7d50 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:24.801855 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fbea0d-faa3-44e0-82fe-c943de07b00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:24.802464 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fbea0d-faa3-44e0-82fe-c943de07b00a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:24.802746 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fbea0d-faa3-44e0-82fe-c943de07b00a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:24.803162 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fbea0d-faa3-44e0-82fe-c943de07b00a None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:24.803810 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fbea0d-faa3-44e0-82fe-c943de07b00a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:24.804113 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fbea0d-faa3-44e0-82fe-c943de07b00a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:24.804945 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d17c42a-4b19-4b59-9344-3026ed64b293 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:24.805474 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fbea0d-faa3-44e0-82fe-c943de07b00a None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:24.814130 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:24.822093 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 522/1031] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:01:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:01:24.823424 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2f5ab-1231-4852-9383-ec7365685e78 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:24.823712 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2f5ab-1231-4852-9383-ec7365685e78 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:24.823985 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2f5ab-1231-4852-9383-ec7365685e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:24.841685 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05766c2e-f0bb-4e40-90c6-1306a55421b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:24.842136 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-bab2f5ab-1231-4852-9383-ec7365685e78 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:24.842285 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05766c2e-f0bb-4e40-90c6-1306a55421b5 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:24.842560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05766c2e-f0bb-4e40-90c6-1306a55421b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:24.842830 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05766c2e-f0bb-4e40-90c6-1306a55421b5 None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:24.843356 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-bab2f5ab-1231-4852-9383-ec7365685e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:24.843482 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05766c2e-f0bb-4e40-90c6-1306a55421b5 None admin] RBAC: Adding query filter params (type=image) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:24.843779 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05766c2e-f0bb-4e40-90c6-1306a55421b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:24.845157 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05766c2e-f0bb-4e40-90c6-1306a55421b5 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:24.849887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 510/1032] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:01:24 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:24.851255 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05766c2e-f0bb-4e40-90c6-1306a55421b5 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:24.912105 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 523/1033] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:24 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:01:24.915041 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab2f5ab-1231-4852-9383-ec7365685e78 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:24.957420 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05766c2e-f0bb-4e40-90c6-1306a55421b5 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:24.973610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:24.991781 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a9ddf1-7af4-425a-9f05-7de9d0397bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:24.992407 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9ddf1-7af4-425a-9f05-7de9d0397bee None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:24.992676 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9ddf1-7af4-425a-9f05-7de9d0397bee None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:24.992939 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9ddf1-7af4-425a-9f05-7de9d0397bee None admin] PATH_INFO: `/v3/users/nova` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:24.999895 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a9ddf1-7af4-425a-9f05-7de9d0397bee None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:25.001311 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a9ddf1-7af4-425a-9f05-7de9d0397bee None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:25.013349 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-55a9ddf1-7af4-425a-9f05-7de9d0397bee None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 14:01:25.013349 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:25.013349 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:25.013349 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:25.013349 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:25.013349 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:25.013349 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:25.013349 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:25.013349 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:25.013349 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:25.013349 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:25.013349 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:25.013349 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:01:25.013349 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:01:25.013349 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:01:25.013349 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:01:25.014332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:25.014332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:25.014332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:01:25.014332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:01:25.014332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:01:25.014332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:01:25.014332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:25.014332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:25.014332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:25.014332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:25.014332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:25.014332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:25.014332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:25.014332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:25.014332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:25.014332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:25.014332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:25.014332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:25.015343 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:25.015343 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:25.015343 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:01:25.015343 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:01:25.015343 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:01:25.015343 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:01:25.015343 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:01:25.015343 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:01:25.015343 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 14:01:25.015343 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:25.015343 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 511/1034] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Sep 12 14:01:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:25.020097 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a9ddf1-7af4-425a-9f05-7de9d0397bee None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:25.020307 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab2f5ab-1231-4852-9383-ec7365685e78 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:25.051583 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:25.055730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae085a-6177-4871-a7aa-eda870ed6bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:25.056642 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae085a-6177-4871-a7aa-eda870ed6bb4 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:25.057701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae085a-6177-4871-a7aa-eda870ed6bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:25.057980 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae085a-6177-4871-a7aa-eda870ed6bb4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:25.058679 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ae085a-6177-4871-a7aa-eda870ed6bb4 None admin] RBAC: Adding query filter params (interface=public, service_id=6f51cb33501b494f886bda2e6ef8d2be, region_id=RegionOne) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:25.059033 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ae085a-6177-4871-a7aa-eda870ed6bb4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:25.060376 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ae085a-6177-4871-a7aa-eda870ed6bb4 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:25.071810 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 524/1035] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Sep 12 14:01:24 2022] GET /identity/v3/endpoints?service_id=6f51cb33501b494f886bda2e6ef8d2be&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:25.073426 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae085a-6177-4871-a7aa-eda870ed6bb4 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:25.134943 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a9ddf1-7af4-425a-9f05-7de9d0397bee None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:25.174473 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3efc7ce-394d-4e74-8d44-67a2e22cf76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:25.175097 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efc7ce-394d-4e74-8d44-67a2e22cf76a None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:25.175370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efc7ce-394d-4e74-8d44-67a2e22cf76a None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:25.175642 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efc7ce-394d-4e74-8d44-67a2e22cf76a None admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:25.176356 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3efc7ce-394d-4e74-8d44-67a2e22cf76a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:25.176653 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3efc7ce-394d-4e74-8d44-67a2e22cf76a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:25.177780 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae085a-6177-4871-a7aa-eda870ed6bb4 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:25.178936 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3efc7ce-394d-4e74-8d44-67a2e22cf76a None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:25.195698 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 512/1036] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Sep 12 14:01:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:01:25.196770 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3efc7ce-394d-4e74-8d44-67a2e22cf76a None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:25.203589 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:25.216114 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93504d2c-570e-411e-b4aa-1e907afdfe75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:25.216711 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93504d2c-570e-411e-b4aa-1e907afdfe75 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:25.216984 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93504d2c-570e-411e-b4aa-1e907afdfe75 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:25.217249 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93504d2c-570e-411e-b4aa-1e907afdfe75 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:25.217849 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93504d2c-570e-411e-b4aa-1e907afdfe75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:25.219204 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93504d2c-570e-411e-b4aa-1e907afdfe75 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:25.226256 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 525/1037] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Sep 12 14:01:25 2022] GET /identity/v3/services => generated 1957 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:01:25.227047 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93504d2c-570e-411e-b4aa-1e907afdfe75 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:25.298220 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:25.309957 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3efc7ce-394d-4e74-8d44-67a2e22cf76a None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:25.346034 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93504d2c-570e-411e-b4aa-1e907afdfe75 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:25.348487 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9f0bc-ea50-47b1-92ed-41c7da85c01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:25.349141 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9f0bc-ea50-47b1-92ed-41c7da85c01f None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:25.349422 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9f0bc-ea50-47b1-92ed-41c7da85c01f None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:25.349697 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9f0bc-ea50-47b1-92ed-41c7da85c01f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:25.351435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f9f0bc-ea50-47b1-92ed-41c7da85c01f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:25.352895 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f9f0bc-ea50-47b1-92ed-41c7da85c01f None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:25.358878 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 513/1038] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:01:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:25.360507 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9f0bc-ea50-47b1-92ed-41c7da85c01f None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:25.390920 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ea801-f2d3-4b7a-a5f3-ff198093277f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:25.391883 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ea801-f2d3-4b7a-a5f3-ff198093277f None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:25.392341 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ea801-f2d3-4b7a-a5f3-ff198093277f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:25.392788 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ea801-f2d3-4b7a-a5f3-ff198093277f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:25.457459 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:25.472205 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9f0bc-ea50-47b1-92ed-41c7da85c01f None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:25.472383 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ea801-f2d3-4b7a-a5f3-ff198093277f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:25.473857 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ea801-f2d3-4b7a-a5f3-ff198093277f None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:25.511543 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67dec1-88e6-4b58-829e-0dfd4e23cc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:25.512602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67dec1-88e6-4b58-829e-0dfd4e23cc54 None admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:25.512602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67dec1-88e6-4b58-829e-0dfd4e23cc54 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:25.512817 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67dec1-88e6-4b58-829e-0dfd4e23cc54 None admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:25.513335 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c67dec1-88e6-4b58-829e-0dfd4e23cc54 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:25.514753 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c67dec1-88e6-4b58-829e-0dfd4e23cc54 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:25.550949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 514/1039] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Sep 12 14:01:25 2022] POST /identity/v3/users => generated 285 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:01:25.587994 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 526/1040] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:01:25 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 14:01:25.594001 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67dec1-88e6-4b58-829e-0dfd4e23cc54 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:25.613868 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:25.708499 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67dec1-88e6-4b58-829e-0dfd4e23cc54 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:25.747393 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566eb1d-71cd-43e9-ab20-a86e77da987d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:25.748129 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566eb1d-71cd-43e9-ab20-a86e77da987d None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:25.748413 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566eb1d-71cd-43e9-ab20-a86e77da987d None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:25.748684 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566eb1d-71cd-43e9-ab20-a86e77da987d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:25.759856 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0566eb1d-71cd-43e9-ab20-a86e77da987d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:25.761293 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0566eb1d-71cd-43e9-ab20-a86e77da987d None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:25.776630 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-0566eb1d-71cd-43e9-ab20-a86e77da987d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:01:25.776630 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:25.776630 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:25.776630 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:25.776630 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:25.776630 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:25.776630 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:25.776630 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:25.776630 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:25.776630 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:25.776630 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:25.776630 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:25.776630 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:01:25.776630 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:01:25.776630 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:01:25.776630 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:01:25.777628 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:25.777628 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:25.777628 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:25.777628 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:25.777628 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:25.777628 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:25.777628 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:25.777628 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:25.777628 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:25.777628 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:25.777628 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:25.777628 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:25.777628 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:25.777628 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:25.777628 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:25.777628 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:25.777628 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:01:25.778538 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:01:25.778538 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:01:25.778538 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:25.778538 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 515/1041] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:01:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:25.783601 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ea801-f2d3-4b7a-a5f3-ff198093277f None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:25.876207 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:25.891507 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ea801-f2d3-4b7a-a5f3-ff198093277f None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:25.927372 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1745341-4cbc-4f43-b09c-e2447d45722a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:25.927981 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1745341-4cbc-4f43-b09c-e2447d45722a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:25.928303 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1745341-4cbc-4f43-b09c-e2447d45722a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:25.928577 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1745341-4cbc-4f43-b09c-e2447d45722a None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:25.929219 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1745341-4cbc-4f43-b09c-e2447d45722a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:25.929519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1745341-4cbc-4f43-b09c-e2447d45722a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:25.930883 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1745341-4cbc-4f43-b09c-e2447d45722a None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:25.948241 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 527/1042] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:01:25.949216 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566eb1d-71cd-43e9-ab20-a86e77da987d None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:26.017209 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:26.053720 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566eb1d-71cd-43e9-ab20-a86e77da987d None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:26.089192 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f9297d-b264-4f57-9086-6697ba542a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:26.089742 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f9297d-b264-4f57-9086-6697ba542a9a None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:26.090018 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f9297d-b264-4f57-9086-6697ba542a9a None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:26.090285 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f9297d-b264-4f57-9086-6697ba542a9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:26.099037 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f9297d-b264-4f57-9086-6697ba542a9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:26.100521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f9297d-b264-4f57-9086-6697ba542a9a None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:26.111831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-d8f9297d-b264-4f57-9086-6697ba542a9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:01:26.111831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:26.111831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:26.111831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:26.111831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:26.111831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:26.111831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:26.111831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:26.111831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:26.111831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:26.111831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:26.111831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:26.111831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:01:26.111831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:01:26.111831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:01:26.112750 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:01:26.112750 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:26.112750 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:26.112750 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:26.112750 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:26.112750 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:26.112750 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:26.112750 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:26.112750 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:26.112750 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:26.112750 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:26.112750 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:26.112750 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:26.112750 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:26.112750 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:26.112750 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:26.112750 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:26.113728 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:01:26.113728 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:01:26.113728 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:01:26.113728 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:01:26.113728 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:01:26.113728 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:01:26.113728 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:01:26.113728 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:26.113728 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 516/1043] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:01:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:01:26.120765 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1745341-4cbc-4f43-b09c-e2447d45722a None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:26.224276 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1745341-4cbc-4f43-b09c-e2447d45722a None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:26.256898 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86337abd-3f53-4a2d-b45a-25ca8558aa66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:26.257505 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86337abd-3f53-4a2d-b45a-25ca8558aa66 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:26.257782 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86337abd-3f53-4a2d-b45a-25ca8558aa66 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:26.258049 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86337abd-3f53-4a2d-b45a-25ca8558aa66 None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:26.258734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86337abd-3f53-4a2d-b45a-25ca8558aa66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:26.259067 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86337abd-3f53-4a2d-b45a-25ca8558aa66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:26.260523 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86337abd-3f53-4a2d-b45a-25ca8558aa66 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:26.273072 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 528/1044] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 12 14:01:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:26.278943 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f9297d-b264-4f57-9086-6697ba542a9a None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:26.382171 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f9297d-b264-4f57-9086-6697ba542a9a None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:26.407090 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:26.420803 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57efa589-8ca2-4050-9ac9-280389a9320f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:26.421413 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57efa589-8ca2-4050-9ac9-280389a9320f None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:26.421691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57efa589-8ca2-4050-9ac9-280389a9320f None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:26.421964 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57efa589-8ca2-4050-9ac9-280389a9320f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:26.422689 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57efa589-8ca2-4050-9ac9-280389a9320f None admin] RBAC: Adding query filter params (role.id=c5ed025905434cacbeed08d80fc31213, scope.project.id=f666a617f8d14eb18bbeb076032b82f2, user.id=854ab2dcec5a48b18f409b33cb61b5da) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:26.423026 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57efa589-8ca2-4050-9ac9-280389a9320f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:26.424440 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57efa589-8ca2-4050-9ac9-280389a9320f None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:26.430541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 517/1045] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 14:01:26 2022] GET /identity/v3/role_assignments?user.id=854ab2dcec5a48b18f409b33cb61b5da&scope.project.id=f666a617f8d14eb18bbeb076032b82f2&role.id=c5ed025905434cacbeed08d80fc31213 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:27.403917 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73180605-3f32-49da-a2df-833a0235ed52 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:27.404444 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73180605-3f32-49da-a2df-833a0235ed52 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:27.404444 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73180605-3f32-49da-a2df-833a0235ed52 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:27.405796 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 529/1046] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:01:27.411238 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daffb39-d197-46f1-b3d1-b63c87a4d553 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:27.411642 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daffb39-d197-46f1-b3d1-b63c87a4d553 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:27.411849 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daffb39-d197-46f1-b3d1-b63c87a4d553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:27.430356 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-1daffb39-d197-46f1-b3d1-b63c87a4d553 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:27.431631 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-1daffb39-d197-46f1-b3d1-b63c87a4d553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:27.504145 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 518/1047] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:27 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:01:27.769033 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73180605-3f32-49da-a2df-833a0235ed52 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:27.776383 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12cc57f-f66b-4477-adef-7fd423b32d76 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:27.776665 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12cc57f-f66b-4477-adef-7fd423b32d76 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:27.776937 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12cc57f-f66b-4477-adef-7fd423b32d76 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:27.778470 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:27.778581 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 519/1048] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 14:01:27.783400 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a04051-be20-4962-a9bd-a3a6951d0563 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:27.783678 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a04051-be20-4962-a9bd-a3a6951d0563 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:27.783947 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a04051-be20-4962-a9bd-a3a6951d0563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:27.801649 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-d1a04051-be20-4962-a9bd-a3a6951d0563 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:27.802822 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-d1a04051-be20-4962-a9bd-a3a6951d0563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:27.879160 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73180605-3f32-49da-a2df-833a0235ed52 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:27.891275 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 520/1049] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:27 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:01:27.917363 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410a8f5a-0da0-43f0-a270-160fbccd7c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:27.917989 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a8f5a-0da0-43f0-a270-160fbccd7c52 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:27.918266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a8f5a-0da0-43f0-a270-160fbccd7c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:27.918534 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a8f5a-0da0-43f0-a270-160fbccd7c52 None admin] PATH_INFO: `/v3/services/image` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:27.919151 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410a8f5a-0da0-43f0-a270-160fbccd7c52 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:27.920491 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410a8f5a-0da0-43f0-a270-160fbccd7c52 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:27.929679 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-410a8f5a-0da0-43f0-a270-160fbccd7c52 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 14:01:27.929679 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:27.929679 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 14:01:27.929679 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 14:01:27.929679 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 14:01:27.929679 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 14:01:27.929679 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 14:01:27.929679 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:01:27.929679 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 14:01:27.929679 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:27.929679 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 14:01:27.929679 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:27.929679 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:27.929679 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:27.929679 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:27.929679 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:27.929679 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:27.930620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:27.930620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:27.930620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:27.930620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:27.930620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:27.930620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:27.930620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 14:01:27.930620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 14:01:27.930620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 14:01:27.930620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 14:01:27.930620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:27.930620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:27.930620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:27.930620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:27.930620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:27.930620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:27.931486 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:27.931486 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:27.931486 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:27.931486 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:27.931486 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:27.931486 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:27.931486 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:27.931486 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:27.931486 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:27.931486 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:27.931486 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 14:01:27.931486 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:01:27.931486 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 14:01:27.931486 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:27.931486 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 530/1050] 10.222.0.42 () {56 vars in 1105 bytes} [Mon Sep 12 14:01:27 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:27.936534 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a04051-be20-4962-a9bd-a3a6951d0563 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:27.989493 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:28.040216 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a04051-be20-4962-a9bd-a3a6951d0563 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:28.074038 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155d083-0c97-4891-a80a-36ff389e90e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:28.074634 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155d083-0c97-4891-a80a-36ff389e90e2 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:28.074933 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155d083-0c97-4891-a80a-36ff389e90e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:28.075202 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155d083-0c97-4891-a80a-36ff389e90e2 None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:28.075820 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c155d083-0c97-4891-a80a-36ff389e90e2 None admin] RBAC: Adding query filter params (name=image) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:28.076114 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c155d083-0c97-4891-a80a-36ff389e90e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:28.077453 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c155d083-0c97-4891-a80a-36ff389e90e2 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:28.086936 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 521/1051] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:01:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:28.088182 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410a8f5a-0da0-43f0-a270-160fbccd7c52 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:28.152939 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155d083-0c97-4891-a80a-36ff389e90e2 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:28.159633 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:28.195573 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410a8f5a-0da0-43f0-a270-160fbccd7c52 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:28.231480 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff922ca-1846-42e0-8e79-cd89f93fae33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:28.232084 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff922ca-1846-42e0-8e79-cd89f93fae33 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:28.232783 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff922ca-1846-42e0-8e79-cd89f93fae33 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:28.233061 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff922ca-1846-42e0-8e79-cd89f93fae33 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:28.233699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff922ca-1846-42e0-8e79-cd89f93fae33 None admin] RBAC: Adding query filter params (type=image) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:28.234001 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff922ca-1846-42e0-8e79-cd89f93fae33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:28.236051 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff922ca-1846-42e0-8e79-cd89f93fae33 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:28.246598 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 531/1052] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:01:28 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:28.247616 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff922ca-1846-42e0-8e79-cd89f93fae33 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:28.263695 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155d083-0c97-4891-a80a-36ff389e90e2 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:28.278421 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:28.300563 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db4c13-19ef-42ec-bb57-9e20541642df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:28.300971 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db4c13-19ef-42ec-bb57-9e20541642df None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:28.301249 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db4c13-19ef-42ec-bb57-9e20541642df None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:28.301520 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db4c13-19ef-42ec-bb57-9e20541642df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:28.312408 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02db4c13-19ef-42ec-bb57-9e20541642df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:01:28.317287 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-02db4c13-19ef-42ec-bb57-9e20541642df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:01:28.317287 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:28.317287 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:28.317287 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:28.317287 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:28.317287 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:28.317287 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:28.317287 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:28.317287 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:28.317287 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:28.317287 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:28.317287 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:28.317287 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:01:28.317287 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:01:28.317287 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:01:28.317287 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:01:28.318214 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:01:28.318214 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:01:28.318214 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:01:28.318214 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:01:28.318214 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:28.318214 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:28.318214 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:28.318214 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:28.318214 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:28.318214 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:28.318214 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:28.318214 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:28.318214 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:28.318214 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:28.318214 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:28.318214 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:28.319164 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:28.319164 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:28.319164 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:28.319164 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:28.319164 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:01:28.319164 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:01:28.319164 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:01:28.319164 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:01:28.319164 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:01:28.319164 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:01:28.319164 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:01:28.319164 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:28.319164 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 522/1053] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 12 14:01:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:01:28.324345 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db4c13-19ef-42ec-bb57-9e20541642df None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:28.358575 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff922ca-1846-42e0-8e79-cd89f93fae33 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:28.395132 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5016530-b7cb-4584-bad3-3a78172a560e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:28.395742 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5016530-b7cb-4584-bad3-3a78172a560e None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:28.396113 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5016530-b7cb-4584-bad3-3a78172a560e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:28.396356 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5016530-b7cb-4584-bad3-3a78172a560e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:28.397523 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5016530-b7cb-4584-bad3-3a78172a560e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:28.398894 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5016530-b7cb-4584-bad3-3a78172a560e None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:28.404359 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:28.431272 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 532/1054] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:01:28 2022] POST /identity/v3/endpoints => generated 327 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:01:28.439046 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db4c13-19ef-42ec-bb57-9e20541642df None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:28.490455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613e303-8e69-4dca-9d1b-a86b9efc9dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:28.491559 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613e303-8e69-4dca-9d1b-a86b9efc9dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:28.492094 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613e303-8e69-4dca-9d1b-a86b9efc9dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:28.492567 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613e303-8e69-4dca-9d1b-a86b9efc9dc2 None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:28.493681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d613e303-8e69-4dca-9d1b-a86b9efc9dc2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:28.494118 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d613e303-8e69-4dca-9d1b-a86b9efc9dc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:28.495814 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d613e303-8e69-4dca-9d1b-a86b9efc9dc2 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:28.505955 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 523/1055] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 14:01:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:01:28.515180 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5016530-b7cb-4584-bad3-3a78172a560e None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:28.587260 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8e3ec-880d-4f27-a3cb-bcd79209dba5 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:28.587564 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8e3ec-880d-4f27-a3cb-bcd79209dba5 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:28.587789 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8e3ec-880d-4f27-a3cb-bcd79209dba5 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:28.589286 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 524/1056] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:01:28.594730 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b191f42e-e7dc-485a-800b-b0e3856c5a2e None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:28.595038 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b191f42e-e7dc-485a-800b-b0e3856c5a2e None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:28.595312 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b191f42e-e7dc-485a-800b-b0e3856c5a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:28.614464 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-b191f42e-e7dc-485a-800b-b0e3856c5a2e None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:28.616228 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-b191f42e-e7dc-485a-800b-b0e3856c5a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:28.631810 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5016530-b7cb-4584-bad3-3a78172a560e None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:28.645433 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:28.671953 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a35e7-c41a-4c33-82e3-68d33b55300f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:28.672583 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a35e7-c41a-4c33-82e3-68d33b55300f None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:28.672858 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a35e7-c41a-4c33-82e3-68d33b55300f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:28.673125 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a35e7-c41a-4c33-82e3-68d33b55300f None admin] PATH_INFO: `/v3/users/ae4ad11e4401491795aeea0eb53c5544` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:28.688136 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534a35e7-c41a-4c33-82e3-68d33b55300f None admin] RBAC: Authorizing `identity:get_user(user_id=ae4ad11e4401491795aeea0eb53c5544)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:28.689545 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534a35e7-c41a-4c33-82e3-68d33b55300f None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:28.691439 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 533/1057] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 14:01:28 2022] GET /identity/v3/users/ae4ad11e4401491795aeea0eb53c5544 => generated 285 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:28.694876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 525/1058] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 14:01:28.697571 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a35e7-c41a-4c33-82e3-68d33b55300f None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:28.747306 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:28.798864 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a35e7-c41a-4c33-82e3-68d33b55300f None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:28.838826 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab69ae-132d-492b-8ff3-16ae7f32c63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:28.839408 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab69ae-132d-492b-8ff3-16ae7f32c63f None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:28.839685 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab69ae-132d-492b-8ff3-16ae7f32c63f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:28.839956 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab69ae-132d-492b-8ff3-16ae7f32c63f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:28.906871 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab69ae-132d-492b-8ff3-16ae7f32c63f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:28.908295 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab69ae-132d-492b-8ff3-16ae7f32c63f None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:28.958735 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:28.965542 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b191f42e-e7dc-485a-800b-b0e3856c5a2e None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:29.014648 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 534/1059] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:01:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 14:01:29.020212 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab69ae-132d-492b-8ff3-16ae7f32c63f None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:29.074866 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b191f42e-e7dc-485a-800b-b0e3856c5a2e None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:29.111638 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7d8a5-3131-4e43-a66b-bc8cd26ff76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:29.112182 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7d8a5-3131-4e43-a66b-bc8cd26ff76d None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:29.112455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7d8a5-3131-4e43-a66b-bc8cd26ff76d None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:29.112718 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7d8a5-3131-4e43-a66b-bc8cd26ff76d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:29.123156 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a7d8a5-3131-4e43-a66b-bc8cd26ff76d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:01:29.123284 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab69ae-132d-492b-8ff3-16ae7f32c63f None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:29.127927 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-d1a7d8a5-3131-4e43-a66b-bc8cd26ff76d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 14:01:29.127927 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:29.127927 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:29.127927 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:29.127927 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:29.127927 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:29.127927 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:29.127927 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:29.127927 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:29.127927 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:29.127927 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:29.127927 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:29.127927 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:01:29.127927 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:01:29.127927 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:01:29.127927 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:01:29.128842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:01:29.128842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:01:29.128842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:01:29.128842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:01:29.128842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:29.128842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:29.128842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:29.128842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:29.128842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:29.128842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:29.128842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:29.128842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:29.128842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:29.128842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:29.128842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:29.128842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:29.129687 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:29.129687 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:29.129687 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:29.129687 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:29.129687 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:01:29.129687 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:01:29.129687 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:01:29.129687 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:01:29.129687 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:01:29.129687 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:01:29.129687 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 14:01:29.129687 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:29.129687 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 526/1060] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Sep 12 14:01:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:01:29.136141 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7d8a5-3131-4e43-a66b-bc8cd26ff76d None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:29.162283 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:29.167269 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d8e58-56e8-490f-86c7-04fbc172b8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:29.167911 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d8e58-56e8-490f-86c7-04fbc172b8f5 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:29.168188 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d8e58-56e8-490f-86c7-04fbc172b8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:29.168456 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d8e58-56e8-490f-86c7-04fbc172b8f5 None admin] PATH_INFO: `/v3/projects/ab5d918c7368498bb7a4133064aed54e` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:29.169568 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d8e58-56e8-490f-86c7-04fbc172b8f5 None admin] RBAC: Authorizing `identity:get_project(project_id=ab5d918c7368498bb7a4133064aed54e)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:29.171056 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d8e58-56e8-490f-86c7-04fbc172b8f5 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:29.172726 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 535/1061] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 14:01:29 2022] GET /identity/v3/projects/ab5d918c7368498bb7a4133064aed54e => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:01:29.179254 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d8e58-56e8-490f-86c7-04fbc172b8f5 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:29.198239 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:29.244235 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7d8a5-3131-4e43-a66b-bc8cd26ff76d None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:29.280607 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81602681-3d4f-499e-afe7-4ce31068eee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:29.281204 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81602681-3d4f-499e-afe7-4ce31068eee6 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:29.281476 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81602681-3d4f-499e-afe7-4ce31068eee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:29.281741 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81602681-3d4f-499e-afe7-4ce31068eee6 None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:29.282462 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81602681-3d4f-499e-afe7-4ce31068eee6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:29.282748 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81602681-3d4f-499e-afe7-4ce31068eee6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:29.283243 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d8e58-56e8-490f-86c7-04fbc172b8f5 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:29.284132 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81602681-3d4f-499e-afe7-4ce31068eee6 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:29.297423 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 527/1062] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Sep 12 14:01:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:01:29.298758 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81602681-3d4f-499e-afe7-4ce31068eee6 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:29.318411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f185d-a593-4952-b9d3-f1ca451cfcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:29.319117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f185d-a593-4952-b9d3-f1ca451cfcc1 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:29.319382 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f185d-a593-4952-b9d3-f1ca451cfcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:29.319649 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f185d-a593-4952-b9d3-f1ca451cfcc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:29.320371 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f185d-a593-4952-b9d3-f1ca451cfcc1 None admin] RBAC: Adding query filter params (role.id=e14564f00daf48a28d4e32da8cd2efad, scope.project.id=ab5d918c7368498bb7a4133064aed54e, user.id=ae4ad11e4401491795aeea0eb53c5544) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:29.320664 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f185d-a593-4952-b9d3-f1ca451cfcc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:29.322337 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f185d-a593-4952-b9d3-f1ca451cfcc1 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:29.328881 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 536/1063] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 14:01:29 2022] GET /identity/v3/role_assignments?user.id=ae4ad11e4401491795aeea0eb53c5544&scope.project.id=ab5d918c7368498bb7a4133064aed54e&role.id=e14564f00daf48a28d4e32da8cd2efad => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:01:29.398230 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:29.416986 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81602681-3d4f-499e-afe7-4ce31068eee6 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:29.451860 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279daf89-aeb8-4690-aaaa-cf66afa09a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:29.452563 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279daf89-aeb8-4690-aaaa-cf66afa09a9d None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:29.452776 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279daf89-aeb8-4690-aaaa-cf66afa09a9d None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:29.453025 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279daf89-aeb8-4690-aaaa-cf66afa09a9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:29.465275 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279daf89-aeb8-4690-aaaa-cf66afa09a9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:29.466762 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279daf89-aeb8-4690-aaaa-cf66afa09a9d None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:29.482803 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-279daf89-aeb8-4690-aaaa-cf66afa09a9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:01:29.482803 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:29.482803 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:29.482803 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:29.482803 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:29.482803 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:29.482803 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:29.482803 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:29.482803 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:29.482803 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:29.482803 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:29.482803 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:29.482803 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:01:29.482803 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:01:29.482803 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:01:29.482803 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:01:29.484024 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:29.484024 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:29.484024 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:29.484024 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:29.484024 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:29.484024 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:29.484024 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:29.484024 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:29.484024 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:29.484024 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:29.484024 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:29.484024 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:29.484024 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:29.484024 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:29.484024 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:29.484024 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:29.484024 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:01:29.484919 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:01:29.484919 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:01:29.484919 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:29.484919 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 528/1064] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:01:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:29.492163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f185d-a593-4952-b9d3-f1ca451cfcc1 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:29.579123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:29.600519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f185d-a593-4952-b9d3-f1ca451cfcc1 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:29.632785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2580b-61b3-4a54-8e15-f1da2a91cc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:29.633395 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2580b-61b3-4a54-8e15-f1da2a91cc35 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:29.633671 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2580b-61b3-4a54-8e15-f1da2a91cc35 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:29.633940 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2580b-61b3-4a54-8e15-f1da2a91cc35 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:29.634571 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d2580b-61b3-4a54-8e15-f1da2a91cc35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:29.634886 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d2580b-61b3-4a54-8e15-f1da2a91cc35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:29.636231 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d2580b-61b3-4a54-8e15-f1da2a91cc35 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:29.648714 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 537/1065] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:01:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:29.654194 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279daf89-aeb8-4690-aaaa-cf66afa09a9d None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:29.757065 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279daf89-aeb8-4690-aaaa-cf66afa09a9d None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:29.790798 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c8b547-21b2-4c31-97e4-2267f34c7fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:29.791454 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8b547-21b2-4c31-97e4-2267f34c7fad None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:29.791729 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8b547-21b2-4c31-97e4-2267f34c7fad None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:29.791997 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8b547-21b2-4c31-97e4-2267f34c7fad None admin] PATH_INFO: `/v3/users/nova` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:29.794731 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:29.800437 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c8b547-21b2-4c31-97e4-2267f34c7fad None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:29.802431 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c8b547-21b2-4c31-97e4-2267f34c7fad None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:29.814111 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-16c8b547-21b2-4c31-97e4-2267f34c7fad None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 14:01:29.814111 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:29.814111 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:29.814111 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:29.814111 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:29.814111 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:29.814111 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:29.814111 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:29.814111 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:29.814111 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:29.814111 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:29.814111 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:29.814111 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:01:29.814111 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:01:29.814111 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:01:29.814111 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:01:29.815096 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:29.815096 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:29.815096 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:01:29.815096 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:01:29.815096 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:01:29.815096 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:01:29.815096 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:29.815096 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:29.815096 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:29.815096 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:29.815096 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:29.815096 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:29.815096 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:29.815096 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:29.815096 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:29.815096 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:29.815096 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:29.815096 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:29.816066 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:29.816066 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:29.816066 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:01:29.816066 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:01:29.816066 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:01:29.816066 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:01:29.816066 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:01:29.816066 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:01:29.816066 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 14:01:29.816066 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:29.816066 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 529/1066] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Sep 12 14:01:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:29.821938 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2580b-61b3-4a54-8e15-f1da2a91cc35 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:29.924689 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2580b-61b3-4a54-8e15-f1da2a91cc35 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:29.957882 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e088b4ba-528a-4060-8b5d-1213c93e9d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:29.958492 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088b4ba-528a-4060-8b5d-1213c93e9d87 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:29.958768 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088b4ba-528a-4060-8b5d-1213c93e9d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:29.959064 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088b4ba-528a-4060-8b5d-1213c93e9d87 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:29.960426 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e088b4ba-528a-4060-8b5d-1213c93e9d87 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:29.960729 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e088b4ba-528a-4060-8b5d-1213c93e9d87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:29.962136 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e088b4ba-528a-4060-8b5d-1213c93e9d87 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:29.970724 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:29.980407 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 538/1067] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Sep 12 14:01:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:29.986233 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c8b547-21b2-4c31-97e4-2267f34c7fad None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:30.087440 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c8b547-21b2-4c31-97e4-2267f34c7fad None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:30.121058 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a0057a-84b9-45fc-9e96-ec414b824034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:30.121683 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0057a-84b9-45fc-9e96-ec414b824034 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:30.121962 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0057a-84b9-45fc-9e96-ec414b824034 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:30.122230 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0057a-84b9-45fc-9e96-ec414b824034 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:30.133210 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a0057a-84b9-45fc-9e96-ec414b824034 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:30.134640 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a0057a-84b9-45fc-9e96-ec414b824034 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:30.148596 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-93a0057a-84b9-45fc-9e96-ec414b824034 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:01:30.148596 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:30.148596 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:30.148596 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:30.148596 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:30.148596 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:30.148596 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:30.148596 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:30.148596 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:30.148596 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:30.148596 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:30.148596 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:30.148596 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:01:30.148596 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:01:30.148596 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:01:30.148596 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:01:30.149558 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:30.149558 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:30.149558 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:30.149558 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:30.149558 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:30.149558 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:30.149558 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:30.149558 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:30.149558 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:30.149558 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:30.149558 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:30.149558 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:30.149558 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:30.149558 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:30.149558 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:30.149558 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:30.149558 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:01:30.150484 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:01:30.150484 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:01:30.150484 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:30.154430 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 530/1068] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:01:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:30.155859 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e088b4ba-528a-4060-8b5d-1213c93e9d87 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:30.259595 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e088b4ba-528a-4060-8b5d-1213c93e9d87 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:30.292648 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb41401-1de1-49bf-998e-85a7cfe150c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:30.293257 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb41401-1de1-49bf-998e-85a7cfe150c9 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:30.293536 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb41401-1de1-49bf-998e-85a7cfe150c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:30.293806 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb41401-1de1-49bf-998e-85a7cfe150c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:30.294444 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb41401-1de1-49bf-998e-85a7cfe150c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:30.294743 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb41401-1de1-49bf-998e-85a7cfe150c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:30.296118 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb41401-1de1-49bf-998e-85a7cfe150c9 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:30.312195 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 539/1069] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:30.313813 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a0057a-84b9-45fc-9e96-ec414b824034 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:30.364353 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:30.418121 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a0057a-84b9-45fc-9e96-ec414b824034 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:30.451486 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e8b713-1953-45d1-9b2c-f065936672c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:30.452109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8b713-1953-45d1-9b2c-f065936672c5 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:30.452671 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8b713-1953-45d1-9b2c-f065936672c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:30.452671 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8b713-1953-45d1-9b2c-f065936672c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:30.459201 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e8b713-1953-45d1-9b2c-f065936672c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:30.460624 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e8b713-1953-45d1-9b2c-f065936672c5 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:30.471605 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-18e8b713-1953-45d1-9b2c-f065936672c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:01:30.471605 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:30.471605 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:30.471605 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:30.471605 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:30.471605 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:30.471605 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:30.471605 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:30.471605 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:30.471605 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:30.471605 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:30.471605 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:30.471605 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:01:30.471605 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:01:30.471605 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:01:30.472507 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:01:30.472507 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:30.472507 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:30.472507 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:30.472507 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:30.472507 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:30.472507 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:30.472507 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:30.472507 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:30.472507 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:30.472507 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:30.472507 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:30.472507 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:30.472507 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:30.472507 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:30.472507 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:30.472507 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:30.473417 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:01:30.473417 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:01:30.473417 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:01:30.473417 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:01:30.473417 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:01:30.473417 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:01:30.473417 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:01:30.473417 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:30.475278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 531/1070] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:01:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:30.479484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb41401-1de1-49bf-998e-85a7cfe150c9 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:30.545794 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:30.604541 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb41401-1de1-49bf-998e-85a7cfe150c9 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:30.638159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb871680-c5f4-42c4-b93c-bd9c16545b9e None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:30.638458 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6662be5-b0d9-478a-8ba4-40596c79bd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:30.638580 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb871680-c5f4-42c4-b93c-bd9c16545b9e None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:30.638697 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb871680-c5f4-42c4-b93c-bd9c16545b9e None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:30.639130 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6662be5-b0d9-478a-8ba4-40596c79bd3e None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:30.639411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6662be5-b0d9-478a-8ba4-40596c79bd3e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:30.639686 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6662be5-b0d9-478a-8ba4-40596c79bd3e None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:30.640290 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 532/1071] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 14:01:30.640411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6662be5-b0d9-478a-8ba4-40596c79bd3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:30.640669 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6662be5-b0d9-478a-8ba4-40596c79bd3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:30.642121 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6662be5-b0d9-478a-8ba4-40596c79bd3e None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:30.644939 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e19a2c-9016-43f1-bd8d-4ee4076a68f0 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:30.645219 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e19a2c-9016-43f1-bd8d-4ee4076a68f0 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:30.645489 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e19a2c-9016-43f1-bd8d-4ee4076a68f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:30.660127 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 540/1072] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 12 14:01:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:01:30.661825 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6662be5-b0d9-478a-8ba4-40596c79bd3e None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:30.663658 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-72e19a2c-9016-43f1-bd8d-4ee4076a68f0 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:30.664818 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-72e19a2c-9016-43f1-bd8d-4ee4076a68f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:30.735976 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 533/1073] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 14:01:30.760888 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6662be5-b0d9-478a-8ba4-40596c79bd3e None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:30.778139 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:30.796044 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3944a-bb7d-49bb-b181-a8a207965b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:30.796639 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3944a-bb7d-49bb-b181-a8a207965b0c None admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:30.796922 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3944a-bb7d-49bb-b181-a8a207965b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:30.797202 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3944a-bb7d-49bb-b181-a8a207965b0c None admin] PATH_INFO: `/v3/projects/f666a617f8d14eb18bbeb076032b82f2/users/854ab2dcec5a48b18f409b33cb61b5da/roles/c5ed025905434cacbeed08d80fc31213` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:30.799670 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3944a-bb7d-49bb-b181-a8a207965b0c None admin] RBAC: Authorizing `identity:create_grant(project_id=f666a617f8d14eb18bbeb076032b82f2, user_id=854ab2dcec5a48b18f409b33cb61b5da, role_id=c5ed025905434cacbeed08d80fc31213)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:30.801095 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3944a-bb7d-49bb-b181-a8a207965b0c None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:30.813848 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 541/1074] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 12 14:01:30 2022] PUT /identity/v3/projects/f666a617f8d14eb18bbeb076032b82f2/users/854ab2dcec5a48b18f409b33cb61b5da/roles/c5ed025905434cacbeed08d80fc31213 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:01:31.010110 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e19a2c-9016-43f1-bd8d-4ee4076a68f0 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:31.050899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:31.113472 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e19a2c-9016-43f1-bd8d-4ee4076a68f0 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:31.159804 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685bb151-2094-4eb2-ad5d-26fab2b52eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:31.160421 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bb151-2094-4eb2-ad5d-26fab2b52eb8 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:31.160699 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bb151-2094-4eb2-ad5d-26fab2b52eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:31.160971 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bb151-2094-4eb2-ad5d-26fab2b52eb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:31.229040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685bb151-2094-4eb2-ad5d-26fab2b52eb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:31.230478 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685bb151-2094-4eb2-ad5d-26fab2b52eb8 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:31.268127 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:31.330974 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 534/1075] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:01:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 14:01:31.337050 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3944a-bb7d-49bb-b181-a8a207965b0c None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:31.439291 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3944a-bb7d-49bb-b181-a8a207965b0c None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:31.473535 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d042c-eda4-4ae6-8ba6-6d2cfd65d1df None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:31.474083 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d042c-eda4-4ae6-8ba6-6d2cfd65d1df None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:31.474083 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d042c-eda4-4ae6-8ba6-6d2cfd65d1df None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:31.475102 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c9462-1d29-4667-84cf-c50e11313eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:31.475602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 535/1076] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 14:01:31.475798 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c9462-1d29-4667-84cf-c50e11313eaf None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:31.476084 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c9462-1d29-4667-84cf-c50e11313eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:31.476647 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c9462-1d29-4667-84cf-c50e11313eaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:31.480561 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9377c-774a-4c99-acac-8a088efe1d97 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:31.480840 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9377c-774a-4c99-acac-8a088efe1d97 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:31.481103 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9377c-774a-4c99-acac-8a088efe1d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:31.490188 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c9462-1d29-4667-84cf-c50e11313eaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:31.491714 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c9462-1d29-4667-84cf-c50e11313eaf None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:31.499740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-f9f9377c-774a-4c99-acac-8a088efe1d97 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:31.500913 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-f9f9377c-774a-4c99-acac-8a088efe1d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:31.511594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-577c9462-1d29-4667-84cf-c50e11313eaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:01:31.511594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:31.511594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:31.511594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:31.511594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:31.511594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:31.511594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:31.511594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:31.511594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:31.511594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:31.511594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:31.511594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:31.511594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:01:31.511594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:01:31.511594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:01:31.511594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:01:31.512636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:31.512636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:31.512636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:31.512636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:31.512636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:31.512636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:31.512636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:31.512636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:31.512636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:31.512636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:31.512636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:31.512636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:31.512636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:31.512636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:31.512636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:31.512636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:31.512636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:01:31.513607 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:01:31.513607 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:01:31.513607 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:31.514145 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 542/1077] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:01:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:31.519474 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c9462-1d29-4667-84cf-c50e11313eaf None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:31.551200 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:31.587306 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 536/1078] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 14:01:31.635654 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c9462-1d29-4667-84cf-c50e11313eaf None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:31.669092 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc3b9d6-cb94-4846-9cd8-ebad76f05553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:31.670055 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3b9d6-cb94-4846-9cd8-ebad76f05553 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:31.670550 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3b9d6-cb94-4846-9cd8-ebad76f05553 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:31.671061 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3b9d6-cb94-4846-9cd8-ebad76f05553 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:31.671910 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc3b9d6-cb94-4846-9cd8-ebad76f05553 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:31.672435 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc3b9d6-cb94-4846-9cd8-ebad76f05553 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:31.673996 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc3b9d6-cb94-4846-9cd8-ebad76f05553 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:31.694929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 543/1079] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:01:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:01:31.853507 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9377c-774a-4c99-acac-8a088efe1d97 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:31.924059 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:31.961637 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9377c-774a-4c99-acac-8a088efe1d97 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:31.998187 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a960e89-f3f7-4f02-bb5d-249620c048b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:31.999141 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a960e89-f3f7-4f02-bb5d-249620c048b4 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:31.999591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a960e89-f3f7-4f02-bb5d-249620c048b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:31.999864 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a960e89-f3f7-4f02-bb5d-249620c048b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:32.011918 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a960e89-f3f7-4f02-bb5d-249620c048b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:01:32.016946 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-4a960e89-f3f7-4f02-bb5d-249620c048b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:01:32.016946 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:32.016946 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:32.016946 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:32.016946 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:32.016946 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:32.016946 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:32.016946 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:32.016946 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:32.016946 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:32.016946 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:32.016946 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:32.016946 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:01:32.016946 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:01:32.016946 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:01:32.016946 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:01:32.018109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:01:32.018109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:01:32.018109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:01:32.018109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:01:32.018109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:32.018109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:32.018109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:32.018109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:32.018109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:32.018109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:32.018109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:32.018109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:32.018109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:32.018109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:32.018109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:32.018109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:32.019249 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:32.019249 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:32.019249 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:32.019249 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:32.019249 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:01:32.019249 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:01:32.019249 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:01:32.019249 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:01:32.019249 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:01:32.019249 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:01:32.019249 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:01:32.019249 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:32.019249 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 537/1080] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 12 14:01:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:32.025438 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc3b9d6-cb94-4846-9cd8-ebad76f05553 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:32.113479 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:32.124125 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc3b9d6-cb94-4846-9cd8-ebad76f05553 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:32.156318 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2d8f2b-4e9e-443c-b886-bbda75659061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:32.156920 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d8f2b-4e9e-443c-b886-bbda75659061 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:32.157199 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d8f2b-4e9e-443c-b886-bbda75659061 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:32.157467 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d8f2b-4e9e-443c-b886-bbda75659061 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:32.158124 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2d8f2b-4e9e-443c-b886-bbda75659061 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:32.158423 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2d8f2b-4e9e-443c-b886-bbda75659061 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:32.159787 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2d8f2b-4e9e-443c-b886-bbda75659061 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:32.168070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 544/1081] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 14:01:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:32.173833 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a960e89-f3f7-4f02-bb5d-249620c048b4 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:32.269472 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a960e89-f3f7-4f02-bb5d-249620c048b4 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:32.294990 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:32.303214 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a060b5e9-3486-4e0b-abf3-f962c73ded92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:32.303824 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a060b5e9-3486-4e0b-abf3-f962c73ded92 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:32.304097 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a060b5e9-3486-4e0b-abf3-f962c73ded92 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:32.304365 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a060b5e9-3486-4e0b-abf3-f962c73ded92 None admin] PATH_INFO: `/v3/users/ae4ad11e4401491795aeea0eb53c5544` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:32.305466 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a060b5e9-3486-4e0b-abf3-f962c73ded92 None admin] RBAC: Authorizing `identity:get_user(user_id=ae4ad11e4401491795aeea0eb53c5544)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:32.306895 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a060b5e9-3486-4e0b-abf3-f962c73ded92 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:32.308481 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 538/1082] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 14:01:32 2022] GET /identity/v3/users/ae4ad11e4401491795aeea0eb53c5544 => generated 285 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:01:32.314147 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2d8f2b-4e9e-443c-b886-bbda75659061 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:32.410634 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2d8f2b-4e9e-443c-b886-bbda75659061 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:32.443129 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7995e04-d7fb-4ae7-a2cd-1687d5ad8932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:32.443773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7995e04-d7fb-4ae7-a2cd-1687d5ad8932 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:32.444050 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7995e04-d7fb-4ae7-a2cd-1687d5ad8932 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:32.444319 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7995e04-d7fb-4ae7-a2cd-1687d5ad8932 None admin] PATH_INFO: `/v3/projects/ab5d918c7368498bb7a4133064aed54e` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:32.445409 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7995e04-d7fb-4ae7-a2cd-1687d5ad8932 None admin] RBAC: Authorizing `identity:get_project(project_id=ab5d918c7368498bb7a4133064aed54e)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:32.447099 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7995e04-d7fb-4ae7-a2cd-1687d5ad8932 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:32.448876 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 545/1083] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 14:01:32 2022] GET /identity/v3/projects/ab5d918c7368498bb7a4133064aed54e => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:01:32.454778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a060b5e9-3486-4e0b-abf3-f962c73ded92 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:32.562137 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a060b5e9-3486-4e0b-abf3-f962c73ded92 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:32.594829 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8054ab14-95f5-4a00-a132-ff9bed857ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:32.595506 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8054ab14-95f5-4a00-a132-ff9bed857ba0 None admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:32.595780 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8054ab14-95f5-4a00-a132-ff9bed857ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:32.596046 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8054ab14-95f5-4a00-a132-ff9bed857ba0 None admin] PATH_INFO: `/v3/projects/ab5d918c7368498bb7a4133064aed54e/users/ae4ad11e4401491795aeea0eb53c5544/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:32.597890 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8054ab14-95f5-4a00-a132-ff9bed857ba0 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab5d918c7368498bb7a4133064aed54e, user_id=ae4ad11e4401491795aeea0eb53c5544, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:32.599328 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8054ab14-95f5-4a00-a132-ff9bed857ba0 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:32.612052 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 539/1084] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 12 14:01:32 2022] PUT /identity/v3/projects/ab5d918c7368498bb7a4133064aed54e/users/ae4ad11e4401491795aeea0eb53c5544/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:01:32.963030 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1135f05-425d-440a-b504-89894cb8c255 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:32.963577 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1135f05-425d-440a-b504-89894cb8c255 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:32.963577 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1135f05-425d-440a-b504-89894cb8c255 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:32.964913 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 546/1085] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:01:32.970351 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2cb4a-6e56-4f53-bf3b-51a1688f4b26 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:32.970664 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2cb4a-6e56-4f53-bf3b-51a1688f4b26 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:32.970961 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2cb4a-6e56-4f53-bf3b-51a1688f4b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:32.989215 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-aac2cb4a-6e56-4f53-bf3b-51a1688f4b26 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:32.990428 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-aac2cb4a-6e56-4f53-bf3b-51a1688f4b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:33.039294 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:33.056882 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 540/1086] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:01:33.323412 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1135f05-425d-440a-b504-89894cb8c255 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:33.344982 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:33.422770 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1135f05-425d-440a-b504-89894cb8c255 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:33.455738 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e96adc-796e-4d59-97cc-9c7d765d74c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:33.456375 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e96adc-796e-4d59-97cc-9c7d765d74c4 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:33.456652 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e96adc-796e-4d59-97cc-9c7d765d74c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:33.456923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e96adc-796e-4d59-97cc-9c7d765d74c4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:33.468495 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e96adc-796e-4d59-97cc-9c7d765d74c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:01:33.473464 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-77e96adc-796e-4d59-97cc-9c7d765d74c4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 14:01:33.473464 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:33.473464 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:33.473464 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:33.473464 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:33.473464 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:33.473464 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:33.473464 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:33.473464 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:33.473464 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:33.473464 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:33.473464 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:33.473464 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:01:33.473464 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:01:33.473464 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:01:33.473464 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:01:33.474570 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:01:33.474570 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:01:33.474570 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:01:33.474570 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:01:33.474570 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:33.474570 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:33.474570 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:33.474570 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:33.474570 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:33.474570 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:33.474570 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:33.474570 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:33.474570 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:33.474570 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:33.474570 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:33.474570 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:33.475986 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:33.475986 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:33.475986 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:33.475986 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:33.475986 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:01:33.475986 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:01:33.475986 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:01:33.475986 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:01:33.475986 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:01:33.475986 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:01:33.475986 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 14:01:33.475986 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:33.477042 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 547/1087] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Sep 12 14:01:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:33.481396 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2cb4a-6e56-4f53-bf3b-51a1688f4b26 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:33.575817 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2cb4a-6e56-4f53-bf3b-51a1688f4b26 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:33.608202 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed1be6-8f32-406e-8a19-2de48e04a69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:33.608789 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed1be6-8f32-406e-8a19-2de48e04a69a None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:33.609062 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed1be6-8f32-406e-8a19-2de48e04a69a None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:33.609334 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed1be6-8f32-406e-8a19-2de48e04a69a None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:33.609978 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ed1be6-8f32-406e-8a19-2de48e04a69a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:33.610271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ed1be6-8f32-406e-8a19-2de48e04a69a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:33.611641 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ed1be6-8f32-406e-8a19-2de48e04a69a None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:33.619818 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 541/1088] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Sep 12 14:01:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:33.625477 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e96adc-796e-4d59-97cc-9c7d765d74c4 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:33.706165 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:33.725802 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e96adc-796e-4d59-97cc-9c7d765d74c4 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:33.758464 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dd9891-1df8-4870-b22b-81c14ba5154e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:33.759391 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd9891-1df8-4870-b22b-81c14ba5154e None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:33.759671 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd9891-1df8-4870-b22b-81c14ba5154e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:33.759943 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd9891-1df8-4870-b22b-81c14ba5154e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:33.769554 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dd9891-1df8-4870-b22b-81c14ba5154e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:33.770999 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dd9891-1df8-4870-b22b-81c14ba5154e None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:33.785637 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-e2dd9891-1df8-4870-b22b-81c14ba5154e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:01:33.785637 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:33.785637 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:33.785637 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:33.785637 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:33.785637 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:33.785637 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:33.785637 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:33.785637 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:33.785637 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:33.785637 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:33.785637 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:33.785637 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:01:33.785637 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:01:33.785637 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:01:33.785637 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:01:33.786773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:33.786773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:33.786773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:33.786773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:33.786773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:33.786773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:33.786773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:33.786773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:33.786773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:33.786773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:33.786773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:33.786773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:33.786773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:33.786773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:33.786773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:33.786773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:33.786773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:01:33.788289 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:01:33.788289 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:01:33.788289 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:33.788289 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 548/1089] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:01:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:33.792959 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed1be6-8f32-406e-8a19-2de48e04a69a None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:33.844236 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:33.894500 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed1be6-8f32-406e-8a19-2de48e04a69a None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:33.927370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8eff65-c489-470d-b1bf-18655d3efed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:33.927990 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8eff65-c489-470d-b1bf-18655d3efed9 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:33.928271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8eff65-c489-470d-b1bf-18655d3efed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:33.928542 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8eff65-c489-470d-b1bf-18655d3efed9 None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:33.929173 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8eff65-c489-470d-b1bf-18655d3efed9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:33.929473 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8eff65-c489-470d-b1bf-18655d3efed9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:33.930827 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8eff65-c489-470d-b1bf-18655d3efed9 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:33.942226 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 542/1090] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:01:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:01:33.948070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dd9891-1df8-4870-b22b-81c14ba5154e None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:34.044512 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dd9891-1df8-4870-b22b-81c14ba5154e None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:34.071806 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:34.077237 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e0602d-125f-4e07-adf2-a6ab2c573b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:34.077880 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e0602d-125f-4e07-adf2-a6ab2c573b5a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:34.078151 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e0602d-125f-4e07-adf2-a6ab2c573b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:34.078414 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e0602d-125f-4e07-adf2-a6ab2c573b5a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:34.086147 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e0602d-125f-4e07-adf2-a6ab2c573b5a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:34.087519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e0602d-125f-4e07-adf2-a6ab2c573b5a None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:34.098183 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-a0e0602d-125f-4e07-adf2-a6ab2c573b5a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 14:01:34.098183 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:34.098183 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:34.098183 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:34.098183 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:34.098183 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:34.098183 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:34.098183 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:34.098183 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:34.098183 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:34.098183 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:34.098183 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:34.098183 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:01:34.098183 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:01:34.098183 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:01:34.098183 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:01:34.104863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:34.104863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:34.104863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:01:34.104863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:01:34.104863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:01:34.104863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:01:34.104863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:34.104863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:34.104863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:34.104863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:34.104863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:34.104863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:34.104863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:34.104863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:34.104863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:34.104863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:34.104863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:34.104863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:34.106077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:34.106077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:34.106077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:01:34.106077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:01:34.106077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:01:34.106077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:01:34.106077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:01:34.106077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:01:34.106077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 14:01:34.106077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:34.106077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 549/1091] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Sep 12 14:01:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:34.106717 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8eff65-c489-470d-b1bf-18655d3efed9 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:34.203004 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8eff65-c489-470d-b1bf-18655d3efed9 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:34.212590 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:34.236757 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620cf173-4997-4279-86e7-274220213139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:34.237383 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620cf173-4997-4279-86e7-274220213139 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:34.237659 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620cf173-4997-4279-86e7-274220213139 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:34.237925 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620cf173-4997-4279-86e7-274220213139 None admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:34.238646 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620cf173-4997-4279-86e7-274220213139 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:34.238982 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620cf173-4997-4279-86e7-274220213139 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:34.240387 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620cf173-4997-4279-86e7-274220213139 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:34.257468 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 543/1092] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Sep 12 14:01:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:34.263136 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e0602d-125f-4e07-adf2-a6ab2c573b5a None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:34.358436 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e0602d-125f-4e07-adf2-a6ab2c573b5a None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:34.390486 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c18d0-7a6a-4f35-be24-31644cf4fb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:34.391103 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c18d0-7a6a-4f35-be24-31644cf4fb2e None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:34.391379 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c18d0-7a6a-4f35-be24-31644cf4fb2e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:34.391646 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c18d0-7a6a-4f35-be24-31644cf4fb2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:34.446797 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:34.456817 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949c18d0-7a6a-4f35-be24-31644cf4fb2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:34.458261 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949c18d0-7a6a-4f35-be24-31644cf4fb2e None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:34.557883 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 550/1093] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:01:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 14:01:34.563992 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620cf173-4997-4279-86e7-274220213139 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:34.665929 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620cf173-4997-4279-86e7-274220213139 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:34.700117 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d7491a-eb08-4f00-8620-a37453640163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:34.700737 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d7491a-eb08-4f00-8620-a37453640163 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:34.701017 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d7491a-eb08-4f00-8620-a37453640163 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:34.701289 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d7491a-eb08-4f00-8620-a37453640163 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:34.711241 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d7491a-eb08-4f00-8620-a37453640163 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:34.712670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d7491a-eb08-4f00-8620-a37453640163 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:34.727040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-24d7491a-eb08-4f00-8620-a37453640163 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:01:34.727040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:34.727040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:34.727040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:34.727040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:34.727040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:34.727040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:34.727040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:34.727040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:34.727040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:34.727040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:34.727040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:34.727040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:01:34.727040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:01:34.727040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:01:34.727040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:01:34.728208 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:34.728208 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:34.728208 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:34.728208 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:34.728208 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:34.728208 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:34.728208 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:34.728208 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:34.728208 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:34.728208 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:34.728208 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:34.728208 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:34.728208 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:34.728208 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:34.728208 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:34.728208 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:34.728208 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:01:34.729329 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:01:34.729329 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:01:34.729329 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:34.729329 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 544/1094] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:01:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:34.731345 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b7956-db65-44cb-abc6-84407863afbf None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:34.731633 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b7956-db65-44cb-abc6-84407863afbf None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:34.731906 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b7956-db65-44cb-abc6-84407863afbf None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:34.733236 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 551/1095] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:01:34.735489 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d7491a-eb08-4f00-8620-a37453640163 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:34.738245 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55dd3e-9e5c-4b46-aeec-928b51ba600c None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:34.738524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55dd3e-9e5c-4b46-aeec-928b51ba600c None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:34.738800 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55dd3e-9e5c-4b46-aeec-928b51ba600c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:34.757437 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-0a55dd3e-9e5c-4b46-aeec-928b51ba600c None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:34.758611 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-0a55dd3e-9e5c-4b46-aeec-928b51ba600c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:34.759831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:34.827909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 552/1096] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 14:01:34.841662 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d7491a-eb08-4f00-8620-a37453640163 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:34.874075 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4cc7a2-61ac-4862-9250-46e09bf693cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:34.874671 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4cc7a2-61ac-4862-9250-46e09bf693cd None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:34.874972 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4cc7a2-61ac-4862-9250-46e09bf693cd None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:34.875238 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4cc7a2-61ac-4862-9250-46e09bf693cd None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:34.875868 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4cc7a2-61ac-4862-9250-46e09bf693cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:34.876164 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4cc7a2-61ac-4862-9250-46e09bf693cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:34.877503 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4cc7a2-61ac-4862-9250-46e09bf693cd None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:34.889172 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 545/1097] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:01:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:34.895143 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a55dd3e-9e5c-4b46-aeec-928b51ba600c None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:34.918483 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:34.990006 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a55dd3e-9e5c-4b46-aeec-928b51ba600c None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:35.021936 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78d464-9fb5-4bf3-bf35-1eb414426d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:35.022547 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78d464-9fb5-4bf3-bf35-1eb414426d48 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:35.022820 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78d464-9fb5-4bf3-bf35-1eb414426d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:35.023112 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78d464-9fb5-4bf3-bf35-1eb414426d48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:35.029374 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab78d464-9fb5-4bf3-bf35-1eb414426d48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:35.030780 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab78d464-9fb5-4bf3-bf35-1eb414426d48 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:35.042441 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-ab78d464-9fb5-4bf3-bf35-1eb414426d48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:01:35.042441 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:35.042441 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:35.042441 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:35.042441 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:35.042441 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:35.042441 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:35.042441 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:35.042441 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:35.042441 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:35.042441 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:35.042441 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:35.042441 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:01:35.042441 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:01:35.042441 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:01:35.043411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:01:35.043411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:35.043411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:35.043411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:35.043411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:35.043411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:35.043411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:35.043411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:35.043411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:35.043411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:35.043411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:35.043411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:35.043411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:35.043411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:35.043411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:35.043411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:35.043411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:35.044303 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:01:35.044303 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:01:35.044303 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:01:35.044303 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:01:35.044303 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:01:35.044303 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:01:35.044303 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:01:35.044303 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:35.048491 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 553/1098] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:01:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:35.049591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4cc7a2-61ac-4862-9250-46e09bf693cd None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:35.096963 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78d464-9fb5-4bf3-bf35-1eb414426d48 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:35.125961 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:35.147050 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4cc7a2-61ac-4862-9250-46e09bf693cd None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:35.177539 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:35.182598 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c13bc74-f16d-437c-91ca-3525e0e852ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:35.183369 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13bc74-f16d-437c-91ca-3525e0e852ba None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:35.183646 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13bc74-f16d-437c-91ca-3525e0e852ba None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:35.183915 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13bc74-f16d-437c-91ca-3525e0e852ba None admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:35.184600 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c13bc74-f16d-437c-91ca-3525e0e852ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:35.184901 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c13bc74-f16d-437c-91ca-3525e0e852ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:35.186318 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c13bc74-f16d-437c-91ca-3525e0e852ba None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:35.196979 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78d464-9fb5-4bf3-bf35-1eb414426d48 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:35.198073 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 546/1099] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 12 14:01:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:35.204077 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c13bc74-f16d-437c-91ca-3525e0e852ba None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:35.229988 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea6a48-d361-4624-8f2a-014d10d0330c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:35.230635 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea6a48-d361-4624-8f2a-014d10d0330c None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:35.230939 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea6a48-d361-4624-8f2a-014d10d0330c None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:35.231209 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea6a48-d361-4624-8f2a-014d10d0330c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:35.241973 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ea6a48-d361-4624-8f2a-014d10d0330c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:01:35.246730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-82ea6a48-d361-4624-8f2a-014d10d0330c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:01:35.246730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:35.246730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:35.246730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:35.246730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:35.246730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:35.246730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:35.246730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:35.246730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:35.246730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:35.246730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:35.246730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:35.246730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:01:35.246730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:01:35.246730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:01:35.246730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:01:35.247891 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:01:35.247891 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:01:35.247891 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:01:35.247891 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:01:35.247891 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:35.247891 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:35.247891 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:35.247891 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:35.247891 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:35.247891 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:35.247891 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:35.247891 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:35.247891 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:35.247891 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:35.247891 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:35.247891 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:35.249012 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:35.249012 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:35.249012 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:35.249012 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:35.249012 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:01:35.249012 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:01:35.249012 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:01:35.249012 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:01:35.249012 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:01:35.249012 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:01:35.249012 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:01:35.249012 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:01:35.249012 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 554/1100] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 12 14:01:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:01:35.253599 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea6a48-d361-4624-8f2a-014d10d0330c None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:35.305578 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c13bc74-f16d-437c-91ca-3525e0e852ba None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:35.338493 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc7870e-d4d6-424f-b249-faf37fa302ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:35.339112 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7870e-d4d6-424f-b249-faf37fa302ba None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:35.339389 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7870e-d4d6-424f-b249-faf37fa302ba None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:35.339659 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7870e-d4d6-424f-b249-faf37fa302ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:35.340373 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc7870e-d4d6-424f-b249-faf37fa302ba None admin] RBAC: Adding query filter params (role.id=c5ed025905434cacbeed08d80fc31213, scope.project.id=f666a617f8d14eb18bbeb076032b82f2, user.id=854ab2dcec5a48b18f409b33cb61b5da) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:35.340672 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc7870e-d4d6-424f-b249-faf37fa302ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:35.342102 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc7870e-d4d6-424f-b249-faf37fa302ba None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:35.347696 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 547/1101] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 14:01:35 2022] GET /identity/v3/role_assignments?user.id=854ab2dcec5a48b18f409b33cb61b5da&scope.project.id=f666a617f8d14eb18bbeb076032b82f2&role.id=c5ed025905434cacbeed08d80fc31213 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:01:35.359096 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea6a48-d361-4624-8f2a-014d10d0330c None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:35.394624 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1cbfbd-1401-477b-9b64-d453a78b6e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:35.395269 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1cbfbd-1401-477b-9b64-d453a78b6e41 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:35.395548 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1cbfbd-1401-477b-9b64-d453a78b6e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:35.395821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1cbfbd-1401-477b-9b64-d453a78b6e41 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:35.397036 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1cbfbd-1401-477b-9b64-d453a78b6e41 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:35.397340 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1cbfbd-1401-477b-9b64-d453a78b6e41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:35.398721 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1cbfbd-1401-477b-9b64-d453a78b6e41 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:35.408741 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 555/1102] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 14:01:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:01:35.415551 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc7870e-d4d6-424f-b249-faf37fa302ba None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:35.418448 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:35.542980 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc7870e-d4d6-424f-b249-faf37fa302ba None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:35.578050 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ac68c-8b16-4326-917f-f3b42f7643e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:35.578683 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ac68c-8b16-4326-917f-f3b42f7643e4 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:35.579022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ac68c-8b16-4326-917f-f3b42f7643e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:35.579298 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ac68c-8b16-4326-917f-f3b42f7643e4 None admin] PATH_INFO: `/v3/users/ae4ad11e4401491795aeea0eb53c5544` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:35.580450 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756ac68c-8b16-4326-917f-f3b42f7643e4 None admin] RBAC: Authorizing `identity:get_user(user_id=ae4ad11e4401491795aeea0eb53c5544)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:35.581870 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756ac68c-8b16-4326-917f-f3b42f7643e4 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:35.583492 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 548/1103] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 14:01:35 2022] GET /identity/v3/users/ae4ad11e4401491795aeea0eb53c5544 => generated 285 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:35.589858 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1cbfbd-1401-477b-9b64-d453a78b6e41 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:35.593793 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:35.689896 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1cbfbd-1401-477b-9b64-d453a78b6e41 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:35.725504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c70227-86ab-495e-a25a-e6c95ab12224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:35.726108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c70227-86ab-495e-a25a-e6c95ab12224 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:35.726388 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c70227-86ab-495e-a25a-e6c95ab12224 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:35.726664 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c70227-86ab-495e-a25a-e6c95ab12224 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:35.795149 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c70227-86ab-495e-a25a-e6c95ab12224 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:35.796506 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c70227-86ab-495e-a25a-e6c95ab12224 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:35.799160 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:35.905835 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 556/1104] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:01:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 12 14:01:35.912142 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ac68c-8b16-4326-917f-f3b42f7643e4 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:35.985840 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:36.017028 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ac68c-8b16-4326-917f-f3b42f7643e4 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:36.050797 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee2059-2d25-4c59-b093-d834520a7d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:36.051381 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee2059-2d25-4c59-b093-d834520a7d36 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:36.051655 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee2059-2d25-4c59-b093-d834520a7d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:36.051924 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee2059-2d25-4c59-b093-d834520a7d36 None admin] PATH_INFO: `/v3/projects/ab5d918c7368498bb7a4133064aed54e` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:36.053580 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee2059-2d25-4c59-b093-d834520a7d36 None admin] RBAC: Authorizing `identity:get_project(project_id=ab5d918c7368498bb7a4133064aed54e)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:36.055050 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee2059-2d25-4c59-b093-d834520a7d36 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:36.056820 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 549/1105] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 14:01:35 2022] GET /identity/v3/projects/ab5d918c7368498bb7a4133064aed54e => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:01:36.063069 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c70227-86ab-495e-a25a-e6c95ab12224 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:36.162551 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c70227-86ab-495e-a25a-e6c95ab12224 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:36.164900 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:36.197790 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041fc48-c947-4366-9143-b50f0a017357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:36.198402 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041fc48-c947-4366-9143-b50f0a017357 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:36.198680 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041fc48-c947-4366-9143-b50f0a017357 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:36.198983 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041fc48-c947-4366-9143-b50f0a017357 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:36.199711 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8041fc48-c947-4366-9143-b50f0a017357 None admin] RBAC: Adding query filter params (role.id=e14564f00daf48a28d4e32da8cd2efad, scope.project.id=ab5d918c7368498bb7a4133064aed54e, user.id=ae4ad11e4401491795aeea0eb53c5544) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:36.200017 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8041fc48-c947-4366-9143-b50f0a017357 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:36.201412 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8041fc48-c947-4366-9143-b50f0a017357 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:36.207153 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 557/1106] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 14:01:36 2022] GET /identity/v3/role_assignments?user.id=ae4ad11e4401491795aeea0eb53c5544&scope.project.id=ab5d918c7368498bb7a4133064aed54e&role.id=e14564f00daf48a28d4e32da8cd2efad => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:38.384589 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c204b62-7372-40bc-97f7-65bf424f2e7c None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:38.385133 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c204b62-7372-40bc-97f7-65bf424f2e7c None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:38.385133 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c204b62-7372-40bc-97f7-65bf424f2e7c None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:38.386441 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 550/1107] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:01:38.391713 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7e1d3-615f-4f11-ace0-10b7a6e00324 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:38.392126 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7e1d3-615f-4f11-ace0-10b7a6e00324 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:38.392259 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7e1d3-615f-4f11-ace0-10b7a6e00324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:38.410453 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-56a7e1d3-615f-4f11-ace0-10b7a6e00324 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:38.411667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-56a7e1d3-615f-4f11-ace0-10b7a6e00324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:38.476137 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 558/1108] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:01:38.756701 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c204b62-7372-40bc-97f7-65bf424f2e7c None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:38.860848 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c204b62-7372-40bc-97f7-65bf424f2e7c None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:38.899854 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69412eeb-2583-4417-a3ca-26e6dc2f4d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:38.900467 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69412eeb-2583-4417-a3ca-26e6dc2f4d51 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:38.900748 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69412eeb-2583-4417-a3ca-26e6dc2f4d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:38.901022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69412eeb-2583-4417-a3ca-26e6dc2f4d51 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:38.902122 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69412eeb-2583-4417-a3ca-26e6dc2f4d51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:38.903646 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69412eeb-2583-4417-a3ca-26e6dc2f4d51 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:38.905210 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 551/1109] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:01:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:38.910923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7e1d3-615f-4f11-ace0-10b7a6e00324 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:39.000851 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:39.004945 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7e1d3-615f-4f11-ace0-10b7a6e00324 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:39.038244 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89716514-0a32-432c-bdd9-219b8b8e90c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:39.038853 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89716514-0a32-432c-bdd9-219b8b8e90c5 None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:39.039144 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89716514-0a32-432c-bdd9-219b8b8e90c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:39.039419 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89716514-0a32-432c-bdd9-219b8b8e90c5 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:39.040071 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89716514-0a32-432c-bdd9-219b8b8e90c5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:39.041480 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89716514-0a32-432c-bdd9-219b8b8e90c5 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:39.062729 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 559/1110] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Sep 12 14:01:38 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:01:41.202066 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d5051-79fd-4750-a19a-4050ae3caae4 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:41.202580 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d5051-79fd-4750-a19a-4050ae3caae4 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:41.202580 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d5051-79fd-4750-a19a-4050ae3caae4 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:41.203879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 552/1111] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:01:41.209584 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352fde1c-d61c-4019-8d75-b8388a86dae9 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:41.210000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352fde1c-d61c-4019-8d75-b8388a86dae9 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:41.210130 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352fde1c-d61c-4019-8d75-b8388a86dae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:41.227791 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-352fde1c-d61c-4019-8d75-b8388a86dae9 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:41.228959 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-352fde1c-d61c-4019-8d75-b8388a86dae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:41.291675 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 560/1112] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:01:41.555744 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d5051-79fd-4750-a19a-4050ae3caae4 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:41.564828 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:41.652659 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d5051-79fd-4750-a19a-4050ae3caae4 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:41.684640 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1e16e-be78-49c0-8f92-153dcd910620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:41.685240 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1e16e-be78-49c0-8f92-153dcd910620 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:41.685511 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1e16e-be78-49c0-8f92-153dcd910620 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:41.685775 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1e16e-be78-49c0-8f92-153dcd910620 None admin] PATH_INFO: `/v3/roles/bcde1b76af8048deb2829c5aaa27296c` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:41.687153 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af1e16e-be78-49c0-8f92-153dcd910620 None admin] RBAC: Authorizing `identity:get_role(role_id=bcde1b76af8048deb2829c5aaa27296c)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:41.688488 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af1e16e-be78-49c0-8f92-153dcd910620 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:41.689767 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 553/1113] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 14:01:41 2022] GET /identity/v3/roles/bcde1b76af8048deb2829c5aaa27296c => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:41.695415 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352fde1c-d61c-4019-8d75-b8388a86dae9 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:41.775718 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:41.791873 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352fde1c-d61c-4019-8d75-b8388a86dae9 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:41.825481 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d23536-ed13-4183-94bc-c6e79e877f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:41.826050 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d23536-ed13-4183-94bc-c6e79e877f67 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:41.826326 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d23536-ed13-4183-94bc-c6e79e877f67 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:41.826593 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d23536-ed13-4183-94bc-c6e79e877f67 None admin] PATH_INFO: `/v3/users/0ecce23fbc6e463c8669f777a8f4f1ba` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:41.839488 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d23536-ed13-4183-94bc-c6e79e877f67 None admin] RBAC: Authorizing `identity:get_user(user_id=0ecce23fbc6e463c8669f777a8f4f1ba)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:41.840906 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d23536-ed13-4183-94bc-c6e79e877f67 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:41.842504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 561/1114] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 14:01:41 2022] GET /identity/v3/users/0ecce23fbc6e463c8669f777a8f4f1ba => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:41.848178 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1e16e-be78-49c0-8f92-153dcd910620 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:41.919993 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:41.945556 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1e16e-be78-49c0-8f92-153dcd910620 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:41.977826 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b27858-215f-4d6c-8495-7f3032b5f497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:41.978409 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b27858-215f-4d6c-8495-7f3032b5f497 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:41.978683 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b27858-215f-4d6c-8495-7f3032b5f497 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:41.978981 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b27858-215f-4d6c-8495-7f3032b5f497 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:42.043931 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b27858-215f-4d6c-8495-7f3032b5f497 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:42.045351 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b27858-215f-4d6c-8495-7f3032b5f497 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:42.115193 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:42.139462 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 554/1115] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:01:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 14:01:42.145453 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d23536-ed13-4183-94bc-c6e79e877f67 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:42.240379 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d23536-ed13-4183-94bc-c6e79e877f67 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:42.272137 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c2cf64-e6b6-433f-8aff-6d1589c87141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:42.272745 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2cf64-e6b6-433f-8aff-6d1589c87141 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:42.273018 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2cf64-e6b6-433f-8aff-6d1589c87141 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:42.273283 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2cf64-e6b6-433f-8aff-6d1589c87141 None admin] PATH_INFO: `/v3/projects/ab5d918c7368498bb7a4133064aed54e` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:42.274356 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c2cf64-e6b6-433f-8aff-6d1589c87141 None admin] RBAC: Authorizing `identity:get_project(project_id=ab5d918c7368498bb7a4133064aed54e)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:42.275779 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c2cf64-e6b6-433f-8aff-6d1589c87141 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:42.277359 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 562/1116] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 14:01:42 2022] GET /identity/v3/projects/ab5d918c7368498bb7a4133064aed54e => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:42.283431 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b27858-215f-4d6c-8495-7f3032b5f497 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:42.377902 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b27858-215f-4d6c-8495-7f3032b5f497 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:42.409989 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab298ccd-121e-4693-8fc5-afdf8d0157e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:42.410569 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab298ccd-121e-4693-8fc5-afdf8d0157e0 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:42.410872 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab298ccd-121e-4693-8fc5-afdf8d0157e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:42.411132 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab298ccd-121e-4693-8fc5-afdf8d0157e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:42.412385 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab298ccd-121e-4693-8fc5-afdf8d0157e0 None admin] RBAC: Adding query filter params (role.id=bcde1b76af8048deb2829c5aaa27296c, scope.project.id=ab5d918c7368498bb7a4133064aed54e, user.id=0ecce23fbc6e463c8669f777a8f4f1ba) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:42.412693 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab298ccd-121e-4693-8fc5-afdf8d0157e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:42.414080 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab298ccd-121e-4693-8fc5-afdf8d0157e0 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:42.419247 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 555/1117] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 14:01:42 2022] GET /identity/v3/role_assignments?user.id=0ecce23fbc6e463c8669f777a8f4f1ba&scope.project.id=ab5d918c7368498bb7a4133064aed54e&role.id=bcde1b76af8048deb2829c5aaa27296c => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:44.506420 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2cfd2d-eeba-454c-b8b1-0205d3f75470 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:44.507003 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2cfd2d-eeba-454c-b8b1-0205d3f75470 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:44.507003 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2cfd2d-eeba-454c-b8b1-0205d3f75470 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:44.508306 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 563/1118] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:01:44.513333 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db383e-73be-48a3-8837-d8cc2d100b2f None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:44.513732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db383e-73be-48a3-8837-d8cc2d100b2f None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:44.513880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db383e-73be-48a3-8837-d8cc2d100b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:44.531491 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-b3db383e-73be-48a3-8837-d8cc2d100b2f None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:44.532660 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-b3db383e-73be-48a3-8837-d8cc2d100b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:44.561033 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:44.598111 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 556/1119] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:01:44.866827 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2cfd2d-eeba-454c-b8b1-0205d3f75470 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:44.878967 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:44.963840 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2cfd2d-eeba-454c-b8b1-0205d3f75470 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:44.996272 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f3298-1cd2-4af8-8d84-e5569e6c573b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:44.996883 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f3298-1cd2-4af8-8d84-e5569e6c573b None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:44.997158 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f3298-1cd2-4af8-8d84-e5569e6c573b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:44.997427 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f3298-1cd2-4af8-8d84-e5569e6c573b None admin] PATH_INFO: `/v3/roles/bcde1b76af8048deb2829c5aaa27296c` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:44.998787 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f3298-1cd2-4af8-8d84-e5569e6c573b None admin] RBAC: Authorizing `identity:get_role(role_id=bcde1b76af8048deb2829c5aaa27296c)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:45.000156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f3298-1cd2-4af8-8d84-e5569e6c573b None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:45.001452 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 564/1120] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 14:01:44 2022] GET /identity/v3/roles/bcde1b76af8048deb2829c5aaa27296c => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:45.007263 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db383e-73be-48a3-8837-d8cc2d100b2f None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:45.103105 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db383e-73be-48a3-8837-d8cc2d100b2f None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:45.135280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e1581-d37d-44a4-81e2-9b3d36d5be09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:45.135891 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e1581-d37d-44a4-81e2-9b3d36d5be09 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:45.136162 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e1581-d37d-44a4-81e2-9b3d36d5be09 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:45.136429 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e1581-d37d-44a4-81e2-9b3d36d5be09 None admin] PATH_INFO: `/v3/users/0ecce23fbc6e463c8669f777a8f4f1ba` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:45.137518 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574e1581-d37d-44a4-81e2-9b3d36d5be09 None admin] RBAC: Authorizing `identity:get_user(user_id=0ecce23fbc6e463c8669f777a8f4f1ba)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:45.138949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574e1581-d37d-44a4-81e2-9b3d36d5be09 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:45.140486 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 557/1121] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 14:01:45 2022] GET /identity/v3/users/0ecce23fbc6e463c8669f777a8f4f1ba => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:45.146674 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f3298-1cd2-4af8-8d84-e5569e6c573b None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:45.221909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:45.243469 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f3298-1cd2-4af8-8d84-e5569e6c573b None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:45.275604 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6406619-b498-4153-84a1-2f237e6982d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:45.276216 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6406619-b498-4153-84a1-2f237e6982d7 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:45.276494 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6406619-b498-4153-84a1-2f237e6982d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:45.276767 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6406619-b498-4153-84a1-2f237e6982d7 None admin] PATH_INFO: `/v3/projects/ab5d918c7368498bb7a4133064aed54e` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:45.277854 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6406619-b498-4153-84a1-2f237e6982d7 None admin] RBAC: Authorizing `identity:get_project(project_id=ab5d918c7368498bb7a4133064aed54e)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:45.279295 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6406619-b498-4153-84a1-2f237e6982d7 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:45.280901 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 565/1122] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 14:01:45 2022] GET /identity/v3/projects/ab5d918c7368498bb7a4133064aed54e => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:45.287444 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e1581-d37d-44a4-81e2-9b3d36d5be09 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:45.320424 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:45.383351 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e1581-d37d-44a4-81e2-9b3d36d5be09 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:45.416547 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584d968d-f321-4641-849b-275fbb309d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:45.417190 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d968d-f321-4641-849b-275fbb309d04 None admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:45.417468 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d968d-f321-4641-849b-275fbb309d04 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:45.417738 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d968d-f321-4641-849b-275fbb309d04 None admin] PATH_INFO: `/v3/projects/ab5d918c7368498bb7a4133064aed54e/users/0ecce23fbc6e463c8669f777a8f4f1ba/roles/bcde1b76af8048deb2829c5aaa27296c` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:45.419866 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584d968d-f321-4641-849b-275fbb309d04 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab5d918c7368498bb7a4133064aed54e, user_id=0ecce23fbc6e463c8669f777a8f4f1ba, role_id=bcde1b76af8048deb2829c5aaa27296c)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:45.421278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584d968d-f321-4641-849b-275fbb309d04 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:45.433488 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 558/1123] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 12 14:01:45 2022] PUT /identity/v3/projects/ab5d918c7368498bb7a4133064aed54e/users/0ecce23fbc6e463c8669f777a8f4f1ba/roles/bcde1b76af8048deb2829c5aaa27296c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:01:47.528413 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fe8dd-640d-4e86-8382-718b9b1b73ce None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:47.528956 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fe8dd-640d-4e86-8382-718b9b1b73ce None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:47.528956 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fe8dd-640d-4e86-8382-718b9b1b73ce None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:47.530264 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 566/1124] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:01:47.535510 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692adbf-dc6f-44fb-a282-c8b52d6f9f85 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:47.535906 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692adbf-dc6f-44fb-a282-c8b52d6f9f85 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:47.536056 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692adbf-dc6f-44fb-a282-c8b52d6f9f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:47.554177 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-b692adbf-dc6f-44fb-a282-c8b52d6f9f85 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:47.555408 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-b692adbf-dc6f-44fb-a282-c8b52d6f9f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:47.619085 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 559/1125] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:01:47.886206 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0fe8dd-640d-4e86-8382-718b9b1b73ce None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:47.982725 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0fe8dd-640d-4e86-8382-718b9b1b73ce None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:48.016760 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754dae55-2ce8-4ac2-9bbe-c2335033a246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:48.017375 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754dae55-2ce8-4ac2-9bbe-c2335033a246 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:48.017656 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754dae55-2ce8-4ac2-9bbe-c2335033a246 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:48.017929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754dae55-2ce8-4ac2-9bbe-c2335033a246 None admin] PATH_INFO: `/v3/roles/bcde1b76af8048deb2829c5aaa27296c` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:48.019309 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754dae55-2ce8-4ac2-9bbe-c2335033a246 None admin] RBAC: Authorizing `identity:get_role(role_id=bcde1b76af8048deb2829c5aaa27296c)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:48.020660 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754dae55-2ce8-4ac2-9bbe-c2335033a246 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:48.022455 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 567/1126] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 14:01:47 2022] GET /identity/v3/roles/bcde1b76af8048deb2829c5aaa27296c => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:48.028118 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692adbf-dc6f-44fb-a282-c8b52d6f9f85 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:48.054276 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:48.132501 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692adbf-dc6f-44fb-a282-c8b52d6f9f85 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:48.170002 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34b175-0f6f-4c34-812d-e75ff87d2307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:48.170620 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34b175-0f6f-4c34-812d-e75ff87d2307 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:48.170948 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34b175-0f6f-4c34-812d-e75ff87d2307 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:48.171220 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34b175-0f6f-4c34-812d-e75ff87d2307 None admin] PATH_INFO: `/v3/users/0ecce23fbc6e463c8669f777a8f4f1ba` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:48.172331 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee34b175-0f6f-4c34-812d-e75ff87d2307 None admin] RBAC: Authorizing `identity:get_user(user_id=0ecce23fbc6e463c8669f777a8f4f1ba)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:48.173748 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee34b175-0f6f-4c34-812d-e75ff87d2307 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:48.175356 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 560/1127] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 14:01:48 2022] GET /identity/v3/users/0ecce23fbc6e463c8669f777a8f4f1ba => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:48.181371 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754dae55-2ce8-4ac2-9bbe-c2335033a246 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:48.196769 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:48.279255 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754dae55-2ce8-4ac2-9bbe-c2335033a246 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:48.311380 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1654ae-f0c3-4e30-af19-159b7db41c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:48.311972 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1654ae-f0c3-4e30-af19-159b7db41c0e None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:48.312245 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1654ae-f0c3-4e30-af19-159b7db41c0e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:48.312511 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1654ae-f0c3-4e30-af19-159b7db41c0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:48.375909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1654ae-f0c3-4e30-af19-159b7db41c0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:48.377309 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1654ae-f0c3-4e30-af19-159b7db41c0e None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:48.390644 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:48.471203 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 568/1128] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:01:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 14:01:48.477117 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34b175-0f6f-4c34-812d-e75ff87d2307 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:48.565042 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:48.572537 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34b175-0f6f-4c34-812d-e75ff87d2307 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:48.604727 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a61631-0cbb-43e1-8b51-41aece55c385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:48.605329 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a61631-0cbb-43e1-8b51-41aece55c385 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:48.605601 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a61631-0cbb-43e1-8b51-41aece55c385 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:48.605869 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a61631-0cbb-43e1-8b51-41aece55c385 None admin] PATH_INFO: `/v3/projects/ab5d918c7368498bb7a4133064aed54e` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:48.606973 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a61631-0cbb-43e1-8b51-41aece55c385 None admin] RBAC: Authorizing `identity:get_project(project_id=ab5d918c7368498bb7a4133064aed54e)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:48.608373 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a61631-0cbb-43e1-8b51-41aece55c385 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:48.609954 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 561/1129] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 14:01:48 2022] GET /identity/v3/projects/ab5d918c7368498bb7a4133064aed54e => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:48.616112 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1654ae-f0c3-4e30-af19-159b7db41c0e None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:48.710714 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1654ae-f0c3-4e30-af19-159b7db41c0e None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:48.723798 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:48.742911 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7797bc06-f68e-48f3-8ae7-944b184fe55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:48.743509 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797bc06-f68e-48f3-8ae7-944b184fe55a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:48.743784 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797bc06-f68e-48f3-8ae7-944b184fe55a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:48.744047 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797bc06-f68e-48f3-8ae7-944b184fe55a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:48.744755 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7797bc06-f68e-48f3-8ae7-944b184fe55a None admin] RBAC: Adding query filter params (role.id=bcde1b76af8048deb2829c5aaa27296c, scope.project.id=ab5d918c7368498bb7a4133064aed54e, user.id=0ecce23fbc6e463c8669f777a8f4f1ba) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:48.745055 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7797bc06-f68e-48f3-8ae7-944b184fe55a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:48.746443 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7797bc06-f68e-48f3-8ae7-944b184fe55a None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:48.751916 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 569/1130] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 14:01:48 2022] GET /identity/v3/role_assignments?user.id=0ecce23fbc6e463c8669f777a8f4f1ba&scope.project.id=ab5d918c7368498bb7a4133064aed54e&role.id=bcde1b76af8048deb2829c5aaa27296c => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:01:50.888083 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c31367-06de-4a09-8863-602655532c21 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:50.888599 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c31367-06de-4a09-8863-602655532c21 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:50.888599 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c31367-06de-4a09-8863-602655532c21 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:50.889938 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 562/1131] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:01:50.895168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4406e08-7d81-41f4-b501-a16f88f414aa None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:50.895541 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4406e08-7d81-41f4-b501-a16f88f414aa None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:50.895712 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4406e08-7d81-41f4-b501-a16f88f414aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:50.914018 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-d4406e08-7d81-41f4-b501-a16f88f414aa None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:50.915212 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d4406e08-7d81-41f4-b501-a16f88f414aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:50.978271 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 570/1132] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:01:51.247661 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c31367-06de-4a09-8863-602655532c21 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:51.344423 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c31367-06de-4a09-8863-602655532c21 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:51.378675 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5480aa3-738f-41c2-a0a2-df1d93962a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:51.379416 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5480aa3-738f-41c2-a0a2-df1d93962a79 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:51.379693 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5480aa3-738f-41c2-a0a2-df1d93962a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:51.379961 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5480aa3-738f-41c2-a0a2-df1d93962a79 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:51.381703 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5480aa3-738f-41c2-a0a2-df1d93962a79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:51.383131 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5480aa3-738f-41c2-a0a2-df1d93962a79 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:51.384670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 563/1133] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:01:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:51.390675 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4406e08-7d81-41f4-b501-a16f88f414aa None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:51.483905 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4406e08-7d81-41f4-b501-a16f88f414aa None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:51.493203 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:51.516920 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ea92ed-d77e-49d6-b487-f8228540a29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:51.517508 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea92ed-d77e-49d6-b487-f8228540a29e None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:51.517793 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea92ed-d77e-49d6-b487-f8228540a29e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:51.518057 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea92ed-d77e-49d6-b487-f8228540a29e None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:51.518696 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ea92ed-d77e-49d6-b487-f8228540a29e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:51.520122 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ea92ed-d77e-49d6-b487-f8228540a29e None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:51.545274 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 571/1134] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Sep 12 14:01:51 2022] POST /identity/v3/projects => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:01:53.674320 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717c5a9-54c2-472f-8e71-b077334ae6a8 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:53.674914 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717c5a9-54c2-472f-8e71-b077334ae6a8 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:53.674914 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717c5a9-54c2-472f-8e71-b077334ae6a8 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:53.676200 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 564/1135] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:01:53.681932 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e163d-da06-4681-a68d-1f50aa038f87 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:53.682349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e163d-da06-4681-a68d-1f50aa038f87 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:53.682479 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e163d-da06-4681-a68d-1f50aa038f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:53.700420 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-5c6e163d-da06-4681-a68d-1f50aa038f87 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:53.701594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-5c6e163d-da06-4681-a68d-1f50aa038f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:53.764593 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 572/1136] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:01:54.035135 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717c5a9-54c2-472f-8e71-b077334ae6a8 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:54.064649 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:54.132813 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717c5a9-54c2-472f-8e71-b077334ae6a8 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:54.165308 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36208497-cf38-453f-930e-ffcf150b974b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:54.165906 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36208497-cf38-453f-930e-ffcf150b974b None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:54.166179 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36208497-cf38-453f-930e-ffcf150b974b None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:54.166446 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36208497-cf38-453f-930e-ffcf150b974b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:54.168022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36208497-cf38-453f-930e-ffcf150b974b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:54.169430 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36208497-cf38-453f-930e-ffcf150b974b None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:54.170980 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 565/1137] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:01:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:54.181529 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e163d-da06-4681-a68d-1f50aa038f87 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:54.268628 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:54.279439 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e163d-da06-4681-a68d-1f50aa038f87 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:54.312332 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581dcb6-2918-416e-a161-257616bcee2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:54.312864 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581dcb6-2918-416e-a161-257616bcee2b None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:54.313136 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581dcb6-2918-416e-a161-257616bcee2b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:54.313401 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581dcb6-2918-416e-a161-257616bcee2b None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:54.314043 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581dcb6-2918-416e-a161-257616bcee2b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:54.315478 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581dcb6-2918-416e-a161-257616bcee2b None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:54.335718 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 573/1138] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Sep 12 14:01:54 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:01:56.507122 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ab149-23ec-4b36-8dd3-d4a26660aca2 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:56.507705 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ab149-23ec-4b36-8dd3-d4a26660aca2 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:56.507705 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ab149-23ec-4b36-8dd3-d4a26660aca2 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:56.508985 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 566/1139] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:01:56.514163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e402fa-d9fb-4a5c-9aee-d11c5857397e None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:56.514545 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e402fa-d9fb-4a5c-9aee-d11c5857397e None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:56.514703 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e402fa-d9fb-4a5c-9aee-d11c5857397e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:56.532332 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-26e402fa-d9fb-4a5c-9aee-d11c5857397e None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:56.533490 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-26e402fa-d9fb-4a5c-9aee-d11c5857397e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:01:56.596654 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 574/1140] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:01:56.869275 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ab149-23ec-4b36-8dd3-d4a26660aca2 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:56.962384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:56.967478 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ab149-23ec-4b36-8dd3-d4a26660aca2 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:57.000580 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beee3387-92c2-4351-a03f-50dd6fb185cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:57.001188 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee3387-92c2-4351-a03f-50dd6fb185cf None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:57.001463 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee3387-92c2-4351-a03f-50dd6fb185cf None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:57.001730 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee3387-92c2-4351-a03f-50dd6fb185cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:57.011419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beee3387-92c2-4351-a03f-50dd6fb185cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:01:57.016253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-beee3387-92c2-4351-a03f-50dd6fb185cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:01:57.016253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:01:57.016253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:01:57.016253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:01:57.016253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:01:57.016253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:01:57.016253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:01:57.016253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:01:57.016253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:01:57.016253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:01:57.016253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:01:57.016253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:01:57.016253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:01:57.016253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:01:57.016253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:01:57.016253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:01:57.017347 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:01:57.017347 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:01:57.017347 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:01:57.017347 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:01:57.017347 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:01:57.017347 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:01:57.017347 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:01:57.017347 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:01:57.017347 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:01:57.017347 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:01:57.017347 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:01:57.017347 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:01:57.017347 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:01:57.017347 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:01:57.017347 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:01:57.017347 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:01:57.018363 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:01:57.018363 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:01:57.018363 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:01:57.018363 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:01:57.018363 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:01:57.018363 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:01:57.018363 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:01:57.018363 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:01:57.018363 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:01:57.018363 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:01:57.018363 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:01:57.018363 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:01:57.018363 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 567/1141] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 12 14:01:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:01:57.023757 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e402fa-d9fb-4a5c-9aee-d11c5857397e None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:57.095875 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:57.119130 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e402fa-d9fb-4a5c-9aee-d11c5857397e None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:57.150985 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c489033-2cba-49d7-9525-5afe9181d8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:57.151605 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c489033-2cba-49d7-9525-5afe9181d8b8 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:57.151878 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c489033-2cba-49d7-9525-5afe9181d8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:57.152143 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c489033-2cba-49d7-9525-5afe9181d8b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:57.152796 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c489033-2cba-49d7-9525-5afe9181d8b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:57.153113 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c489033-2cba-49d7-9525-5afe9181d8b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:57.154447 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c489033-2cba-49d7-9525-5afe9181d8b8 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:57.162622 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 575/1142] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 14:01:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:01:57.168376 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beee3387-92c2-4351-a03f-50dd6fb185cf None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:57.263685 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beee3387-92c2-4351-a03f-50dd6fb185cf None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:57.297108 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9b5bc-2da8-4ef9-94be-d9daa2ab8b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:57.297712 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9b5bc-2da8-4ef9-94be-d9daa2ab8b54 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:57.297986 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9b5bc-2da8-4ef9-94be-d9daa2ab8b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:57.298254 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9b5bc-2da8-4ef9-94be-d9daa2ab8b54 None admin] PATH_INFO: `/v3/users/3e801db406434e9c93d17edcb8a69c60` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:57.310705 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e9b5bc-2da8-4ef9-94be-d9daa2ab8b54 None admin] RBAC: Authorizing `identity:get_user(user_id=3e801db406434e9c93d17edcb8a69c60)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:57.312153 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e9b5bc-2da8-4ef9-94be-d9daa2ab8b54 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:57.313693 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 568/1143] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 14:01:57 2022] GET /identity/v3/users/3e801db406434e9c93d17edcb8a69c60 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:57.319292 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c489033-2cba-49d7-9525-5afe9181d8b8 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:57.416144 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c489033-2cba-49d7-9525-5afe9181d8b8 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:57.448609 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c22c5-cf40-49ca-9df6-fc91fb6cd92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:57.449199 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c22c5-cf40-49ca-9df6-fc91fb6cd92b None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:57.449480 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c22c5-cf40-49ca-9df6-fc91fb6cd92b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:57.449754 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c22c5-cf40-49ca-9df6-fc91fb6cd92b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:57.457896 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:57.515036 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226c22c5-cf40-49ca-9df6-fc91fb6cd92b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:57.516459 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226c22c5-cf40-49ca-9df6-fc91fb6cd92b None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:57.609033 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 576/1144] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:01:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 14:01:57.615094 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9b5bc-2da8-4ef9-94be-d9daa2ab8b54 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:57.632833 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:57.711872 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9b5bc-2da8-4ef9-94be-d9daa2ab8b54 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:57.743864 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a34e5-90af-4263-8031-c9f879303352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:57.744503 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a34e5-90af-4263-8031-c9f879303352 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:57.744782 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a34e5-90af-4263-8031-c9f879303352 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:57.745052 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a34e5-90af-4263-8031-c9f879303352 None admin] PATH_INFO: `/v3/projects/1bc392ca4b78438fb32d28fa66b5665b` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:57.746153 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a34e5-90af-4263-8031-c9f879303352 None admin] RBAC: Authorizing `identity:get_project(project_id=1bc392ca4b78438fb32d28fa66b5665b)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:57.747586 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a34e5-90af-4263-8031-c9f879303352 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:57.749187 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 569/1145] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 14:01:57 2022] GET /identity/v3/projects/1bc392ca4b78438fb32d28fa66b5665b => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:57.755639 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c22c5-cf40-49ca-9df6-fc91fb6cd92b None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:01:57.792948 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:01:57.850670 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c22c5-cf40-49ca-9df6-fc91fb6cd92b None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:01:57.883257 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46efc095-758a-4ca1-96ff-f32d2038740e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:01:57.883839 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46efc095-758a-4ca1-96ff-f32d2038740e None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:57.884114 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46efc095-758a-4ca1-96ff-f32d2038740e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:57.884382 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46efc095-758a-4ca1-96ff-f32d2038740e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:57.885088 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46efc095-758a-4ca1-96ff-f32d2038740e None admin] RBAC: Adding query filter params (role.id=e14564f00daf48a28d4e32da8cd2efad, scope.project.id=1bc392ca4b78438fb32d28fa66b5665b, user.id=3e801db406434e9c93d17edcb8a69c60) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:01:57.885386 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46efc095-758a-4ca1-96ff-f32d2038740e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:01:57.886778 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46efc095-758a-4ca1-96ff-f32d2038740e None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:01:57.891899 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 577/1146] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 14:01:57 2022] GET /identity/v3/role_assignments?user.id=3e801db406434e9c93d17edcb8a69c60&scope.project.id=1bc392ca4b78438fb32d28fa66b5665b&role.id=e14564f00daf48a28d4e32da8cd2efad => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:01:59.969113 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268d24a-dcea-4ca7-815a-0e718b583d36 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:59.969633 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268d24a-dcea-4ca7-815a-0e718b583d36 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:59.969633 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268d24a-dcea-4ca7-815a-0e718b583d36 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:59.970982 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 570/1147] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:01:59.976646 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2e5df-e25e-4e8b-a479-fc67213945b1 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:01:59.977082 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2e5df-e25e-4e8b-a479-fc67213945b1 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:01:59.977176 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2e5df-e25e-4e8b-a479-fc67213945b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:01:59.996113 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-d9e2e5df-e25e-4e8b-a479-fc67213945b1 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:01:59.997287 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d9e2e5df-e25e-4e8b-a479-fc67213945b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:02:00.059982 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 578/1148] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:01:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 14:02:00.325467 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8268d24a-dcea-4ca7-815a-0e718b583d36 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:02:00.405793 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:02:00.422004 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8268d24a-dcea-4ca7-815a-0e718b583d36 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:02:00.455895 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa1d9d6-1da8-426e-ab4c-c6c1037fe5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:02:00.456504 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1d9d6-1da8-426e-ab4c-c6c1037fe5cc None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:00.456777 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1d9d6-1da8-426e-ab4c-c6c1037fe5cc None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:00.457039 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1d9d6-1da8-426e-ab4c-c6c1037fe5cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:00.466611 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa1d9d6-1da8-426e-ab4c-c6c1037fe5cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:02:00.471397 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-1fa1d9d6-1da8-426e-ab4c-c6c1037fe5cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:02:00.471397 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:02:00.471397 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:02:00.471397 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:02:00.471397 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:02:00.471397 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:02:00.471397 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:02:00.471397 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:02:00.471397 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:02:00.471397 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:02:00.471397 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:02:00.471397 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:02:00.471397 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:02:00.471397 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:02:00.471397 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:02:00.471397 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:02:00.472468 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:02:00.472468 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:02:00.472468 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:02:00.472468 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:02:00.472468 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:02:00.472468 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:02:00.472468 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:02:00.472468 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:02:00.472468 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:02:00.472468 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:02:00.472468 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:02:00.472468 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:02:00.472468 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:02:00.472468 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:02:00.472468 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:02:00.472468 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:02:00.473339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:02:00.473339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:02:00.473339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:02:00.473339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:02:00.473339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:02:00.473339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:02:00.473339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:02:00.473339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:02:00.473339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:02:00.473339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:02:00.473339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:02:00.473339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:02:00.473339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 571/1149] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 12 14:02:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:02:00.479056 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2e5df-e25e-4e8b-a479-fc67213945b1 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:02:00.508316 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:02:00.573734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2e5df-e25e-4e8b-a479-fc67213945b1 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:02:00.605809 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc405b-fbf6-4034-b1f5-784e5796dee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:02:00.606349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc405b-fbf6-4034-b1f5-784e5796dee9 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:00.606624 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc405b-fbf6-4034-b1f5-784e5796dee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:00.606915 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc405b-fbf6-4034-b1f5-784e5796dee9 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:00.607569 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fc405b-fbf6-4034-b1f5-784e5796dee9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:02:00.607867 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fc405b-fbf6-4034-b1f5-784e5796dee9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:02:00.609203 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fc405b-fbf6-4034-b1f5-784e5796dee9 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:02:00.617405 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 579/1150] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 14:02:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:02:00.623073 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa1d9d6-1da8-426e-ab4c-c6c1037fe5cc None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:02:00.722558 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa1d9d6-1da8-426e-ab4c-c6c1037fe5cc None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:02:00.756252 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93990795-5495-4caa-8066-ca353fac78ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:02:00.756915 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93990795-5495-4caa-8066-ca353fac78ac None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:00.757194 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93990795-5495-4caa-8066-ca353fac78ac None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:00.757464 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93990795-5495-4caa-8066-ca353fac78ac None admin] PATH_INFO: `/v3/users/3e801db406434e9c93d17edcb8a69c60` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:00.758686 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93990795-5495-4caa-8066-ca353fac78ac None admin] RBAC: Authorizing `identity:get_user(user_id=3e801db406434e9c93d17edcb8a69c60)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:02:00.760195 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93990795-5495-4caa-8066-ca353fac78ac None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:02:00.761763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 572/1151] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 14:02:00 2022] GET /identity/v3/users/3e801db406434e9c93d17edcb8a69c60 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:02:00.767514 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc405b-fbf6-4034-b1f5-784e5796dee9 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:02:00.858731 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:02:00.865707 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc405b-fbf6-4034-b1f5-784e5796dee9 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:02:00.897477 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e67d9-9ec3-427c-9c03-68a144b83613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:02:00.898089 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e67d9-9ec3-427c-9c03-68a144b83613 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:00.898364 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e67d9-9ec3-427c-9c03-68a144b83613 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:00.898635 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e67d9-9ec3-427c-9c03-68a144b83613 None admin] PATH_INFO: `/v3/projects/1bc392ca4b78438fb32d28fa66b5665b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:00.899749 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e67d9-9ec3-427c-9c03-68a144b83613 None admin] RBAC: Authorizing `identity:get_project(project_id=1bc392ca4b78438fb32d28fa66b5665b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:02:00.901150 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e67d9-9ec3-427c-9c03-68a144b83613 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:02:00.902755 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 580/1152] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 14:02:00 2022] GET /identity/v3/projects/1bc392ca4b78438fb32d28fa66b5665b => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:02:00.908698 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93990795-5495-4caa-8066-ca353fac78ac None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:02:00.923669 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:02:01.005926 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93990795-5495-4caa-8066-ca353fac78ac None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:02:01.039168 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0e0a2-7308-4d32-9fd5-e6f71af8b2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:02:01.039811 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0e0a2-7308-4d32-9fd5-e6f71af8b2bf None admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:01.040092 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0e0a2-7308-4d32-9fd5-e6f71af8b2bf None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:01.040365 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0e0a2-7308-4d32-9fd5-e6f71af8b2bf None admin] PATH_INFO: `/v3/projects/1bc392ca4b78438fb32d28fa66b5665b/users/3e801db406434e9c93d17edcb8a69c60/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:01.042234 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b0e0a2-7308-4d32-9fd5-e6f71af8b2bf None admin] RBAC: Authorizing `identity:create_grant(project_id=1bc392ca4b78438fb32d28fa66b5665b, user_id=3e801db406434e9c93d17edcb8a69c60, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:02:01.043668 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b0e0a2-7308-4d32-9fd5-e6f71af8b2bf None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:02:01.055722 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 573/1153] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 12 14:02:00 2022] PUT /identity/v3/projects/1bc392ca4b78438fb32d28fa66b5665b/users/3e801db406434e9c93d17edcb8a69c60/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:02:03.158145 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed6e70-b558-4efd-9ee4-a5867e3e8148 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:03.158694 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed6e70-b558-4efd-9ee4-a5867e3e8148 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:03.158694 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed6e70-b558-4efd-9ee4-a5867e3e8148 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:03.159973 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 581/1154] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:02:03.165313 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a394994-8525-4917-bfac-fa59306b62f8 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:03.165772 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a394994-8525-4917-bfac-fa59306b62f8 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:03.165878 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a394994-8525-4917-bfac-fa59306b62f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:03.184132 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-5a394994-8525-4917-bfac-fa59306b62f8 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:02:03.185299 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-5a394994-8525-4917-bfac-fa59306b62f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:02:03.249951 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 574/1155] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:02:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:02:03.520739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed6e70-b558-4efd-9ee4-a5867e3e8148 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:02:03.620217 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed6e70-b558-4efd-9ee4-a5867e3e8148 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:02:03.651628 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54832f1-a378-459d-b457-f8e6cdb13668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:02:03.652239 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54832f1-a378-459d-b457-f8e6cdb13668 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:03.652518 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54832f1-a378-459d-b457-f8e6cdb13668 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:03.652788 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54832f1-a378-459d-b457-f8e6cdb13668 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:03.662946 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54832f1-a378-459d-b457-f8e6cdb13668 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:02:03.667699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-d54832f1-a378-459d-b457-f8e6cdb13668 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:02:03.667699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:02:03.667699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:02:03.667699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:02:03.667699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:02:03.667699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:02:03.667699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:02:03.667699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:02:03.667699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:02:03.667699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:02:03.667699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:02:03.667699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:02:03.667699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:02:03.667699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:02:03.667699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:02:03.667699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:02:03.668855 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:02:03.668855 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:02:03.668855 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:02:03.668855 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:02:03.668855 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:02:03.668855 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:02:03.668855 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:02:03.668855 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:02:03.668855 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:02:03.668855 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:02:03.668855 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:02:03.668855 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:02:03.668855 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:02:03.668855 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:02:03.668855 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:02:03.668855 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:02:03.669933 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:02:03.669933 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:02:03.669933 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:02:03.669933 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:02:03.669933 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:02:03.669933 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:02:03.669933 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:02:03.669933 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:02:03.669933 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:02:03.669933 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:02:03.669933 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:02:03.669933 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:02:03.669933 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 582/1156] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 12 14:02:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:02:03.674748 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a394994-8525-4917-bfac-fa59306b62f8 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:02:03.677497 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:02:03.772053 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a394994-8525-4917-bfac-fa59306b62f8 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:02:03.804545 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c64e08-e0e2-4337-9370-9f82f020bfeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:02:03.805140 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c64e08-e0e2-4337-9370-9f82f020bfeb None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:03.805418 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c64e08-e0e2-4337-9370-9f82f020bfeb None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:03.805688 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c64e08-e0e2-4337-9370-9f82f020bfeb None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:03.806339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c64e08-e0e2-4337-9370-9f82f020bfeb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:02:03.806637 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c64e08-e0e2-4337-9370-9f82f020bfeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:02:03.808020 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c64e08-e0e2-4337-9370-9f82f020bfeb None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:02:03.815865 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 575/1157] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 14:02:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:02:03.822051 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54832f1-a378-459d-b457-f8e6cdb13668 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:02:03.846553 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:02:03.919375 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54832f1-a378-459d-b457-f8e6cdb13668 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:02:03.951334 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b590a2-264d-4778-b23e-cad33dbc6434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:02:03.951947 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b590a2-264d-4778-b23e-cad33dbc6434 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:03.952221 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b590a2-264d-4778-b23e-cad33dbc6434 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:03.952490 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b590a2-264d-4778-b23e-cad33dbc6434 None admin] PATH_INFO: `/v3/users/3e801db406434e9c93d17edcb8a69c60` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:03.953580 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b590a2-264d-4778-b23e-cad33dbc6434 None admin] RBAC: Authorizing `identity:get_user(user_id=3e801db406434e9c93d17edcb8a69c60)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:02:03.955036 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b590a2-264d-4778-b23e-cad33dbc6434 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:02:03.956587 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 583/1158] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 14:02:03 2022] GET /identity/v3/users/3e801db406434e9c93d17edcb8a69c60 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:02:03.962903 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c64e08-e0e2-4337-9370-9f82f020bfeb None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:02:04.022478 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:02:04.059069 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c64e08-e0e2-4337-9370-9f82f020bfeb None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:02:04.090863 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7242452-c84c-4358-aa77-5bd854684d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:02:04.091453 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7242452-c84c-4358-aa77-5bd854684d4f None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:04.091727 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7242452-c84c-4358-aa77-5bd854684d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:04.091993 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7242452-c84c-4358-aa77-5bd854684d4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:04.155027 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7242452-c84c-4358-aa77-5bd854684d4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:02:04.156446 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7242452-c84c-4358-aa77-5bd854684d4f None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:02:04.213749 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:02:04.249930 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 576/1159] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:02:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 14:02:04.256280 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b590a2-264d-4778-b23e-cad33dbc6434 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:02:04.343302 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:02:04.352777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b590a2-264d-4778-b23e-cad33dbc6434 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:02:04.384934 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c3a642-f88b-4143-8e48-8f0eab413378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:02:04.385544 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3a642-f88b-4143-8e48-8f0eab413378 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:04.385819 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3a642-f88b-4143-8e48-8f0eab413378 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:04.386088 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3a642-f88b-4143-8e48-8f0eab413378 None admin] PATH_INFO: `/v3/projects/1bc392ca4b78438fb32d28fa66b5665b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:04.387198 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c3a642-f88b-4143-8e48-8f0eab413378 None admin] RBAC: Authorizing `identity:get_project(project_id=1bc392ca4b78438fb32d28fa66b5665b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:02:04.388601 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c3a642-f88b-4143-8e48-8f0eab413378 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:02:04.390192 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 584/1160] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 14:02:04 2022] GET /identity/v3/projects/1bc392ca4b78438fb32d28fa66b5665b => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:02:04.396675 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7242452-c84c-4358-aa77-5bd854684d4f None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:02:04.489812 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7242452-c84c-4358-aa77-5bd854684d4f None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:02:04.524258 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e49ac-1544-4337-a1da-330250db6afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:02:04.524841 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e49ac-1544-4337-a1da-330250db6afb None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:04.525116 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e49ac-1544-4337-a1da-330250db6afb None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:04.525384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e49ac-1544-4337-a1da-330250db6afb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:04.526091 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73e49ac-1544-4337-a1da-330250db6afb None admin] RBAC: Adding query filter params (role.id=e14564f00daf48a28d4e32da8cd2efad, scope.project.id=1bc392ca4b78438fb32d28fa66b5665b, user.id=3e801db406434e9c93d17edcb8a69c60) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:02:04.526393 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73e49ac-1544-4337-a1da-330250db6afb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:02:04.527808 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73e49ac-1544-4337-a1da-330250db6afb None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:02:04.533263 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 577/1161] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 14:02:04 2022] GET /identity/v3/role_assignments?user.id=3e801db406434e9c93d17edcb8a69c60&scope.project.id=1bc392ca4b78438fb32d28fa66b5665b&role.id=e14564f00daf48a28d4e32da8cd2efad => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:02:06.652753 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e32a10-aab6-4f34-98d5-86387f46287a None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:06.653280 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e32a10-aab6-4f34-98d5-86387f46287a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:06.653280 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e32a10-aab6-4f34-98d5-86387f46287a None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:06.654625 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 585/1162] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:02:06.659879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180e8e9-e121-46ba-b039-1b55855266e2 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:06.660290 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180e8e9-e121-46ba-b039-1b55855266e2 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:06.660418 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180e8e9-e121-46ba-b039-1b55855266e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:06.679079 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-d180e8e9-e121-46ba-b039-1b55855266e2 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:02:06.680240 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-d180e8e9-e121-46ba-b039-1b55855266e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:02:06.704174 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:02:06.747022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 578/1163] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:02:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:02:07.010903 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e32a10-aab6-4f34-98d5-86387f46287a None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:02:07.107517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e32a10-aab6-4f34-98d5-86387f46287a None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:02:07.138582 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0de840-a2cd-4310-82c2-a4b8541ff6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:02:07.139199 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0de840-a2cd-4310-82c2-a4b8541ff6fc None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:07.139471 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0de840-a2cd-4310-82c2-a4b8541ff6fc None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:07.139735 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0de840-a2cd-4310-82c2-a4b8541ff6fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:07.165483 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:02:07.204395 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0de840-a2cd-4310-82c2-a4b8541ff6fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:02:07.205803 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0de840-a2cd-4310-82c2-a4b8541ff6fc None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:02:07.297428 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 586/1164] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:02:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 14:02:07.303275 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d180e8e9-e121-46ba-b039-1b55855266e2 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:02:07.398637 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d180e8e9-e121-46ba-b039-1b55855266e2 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:02:07.430335 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc7abb4-2b74-4ecf-9456-0da0c2570d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:02:07.430973 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7abb4-2b74-4ecf-9456-0da0c2570d67 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:07.431251 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7abb4-2b74-4ecf-9456-0da0c2570d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:07.431519 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7abb4-2b74-4ecf-9456-0da0c2570d67 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:07.437910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc7abb4-2b74-4ecf-9456-0da0c2570d67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:02:07.439384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc7abb4-2b74-4ecf-9456-0da0c2570d67 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:02:07.451316 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-dcc7abb4-2b74-4ecf-9456-0da0c2570d67 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 12 14:02:07.451316 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:02:07.451316 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 12 14:02:07.451316 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 12 14:02:07.451316 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:02:07.451316 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:02:07.451316 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:02:07.451316 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:02:07.451316 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 12 14:02:07.451316 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:02:07.451316 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 14:02:07.451316 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:02:07.451316 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:02:07.451316 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:02:07.451316 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:02:07.451316 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:02:07.451316 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:02:07.452670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:02:07.452670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:02:07.452670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:02:07.452670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:02:07.452670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:02:07.452670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:02:07.452670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:02:07.452670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:02:07.452670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:02:07.452670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:02:07.452670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:02:07.452670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:02:07.452670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:02:07.452670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:02:07.452670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:02:07.452670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:02:07.453755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:02:07.453755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:02:07.453755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:02:07.453755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:02:07.453755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:02:07.453755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:02:07.453755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:02:07.453755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:02:07.453755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:02:07.453755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:02:07.453755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 12 14:02:07.453755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:02:07.453755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 12 14:02:07.453755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:02:07.453755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 579/1165] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Sep 12 14:02:07 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:02:07.458856 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0de840-a2cd-4310-82c2-a4b8541ff6fc None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:02:07.513189 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:02:07.553539 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0de840-a2cd-4310-82c2-a4b8541ff6fc None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:02:07.586029 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fcc3e7-88d4-462c-bfbf-f48c7f1c4e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:02:07.586613 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fcc3e7-88d4-462c-bfbf-f48c7f1c4e35 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:07.586920 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fcc3e7-88d4-462c-bfbf-f48c7f1c4e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:07.587183 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fcc3e7-88d4-462c-bfbf-f48c7f1c4e35 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:07.587811 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fcc3e7-88d4-462c-bfbf-f48c7f1c4e35 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:02:07.588104 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fcc3e7-88d4-462c-bfbf-f48c7f1c4e35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:02:07.589431 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fcc3e7-88d4-462c-bfbf-f48c7f1c4e35 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:02:07.596187 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 587/1166] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Sep 12 14:02:07 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:02:09.687669 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858aeda-249a-4dc3-98cc-1c9f5315a655 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:09.688198 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858aeda-249a-4dc3-98cc-1c9f5315a655 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:09.688198 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858aeda-249a-4dc3-98cc-1c9f5315a655 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:09.689539 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 580/1167] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:02:09.694657 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c03d2a-ab71-4120-a6f2-73875cb0b833 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:09.695070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c03d2a-ab71-4120-a6f2-73875cb0b833 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:09.695233 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c03d2a-ab71-4120-a6f2-73875cb0b833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:09.713103 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-f8c03d2a-ab71-4120-a6f2-73875cb0b833 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:02:09.714273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f8c03d2a-ab71-4120-a6f2-73875cb0b833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:02:09.777233 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 588/1168] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:02:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:02:10.042356 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3858aeda-249a-4dc3-98cc-1c9f5315a655 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:02:10.061134 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:02:10.144253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3858aeda-249a-4dc3-98cc-1c9f5315a655 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:02:10.177893 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f60cb-f80e-49b8-a134-ec1703e57d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:02:10.178482 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f60cb-f80e-49b8-a134-ec1703e57d41 None admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:10.178757 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f60cb-f80e-49b8-a134-ec1703e57d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:10.179051 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f60cb-f80e-49b8-a134-ec1703e57d41 None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:10.179719 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f60cb-f80e-49b8-a134-ec1703e57d41 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:02:10.181059 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f60cb-f80e-49b8-a134-ec1703e57d41 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:02:10.198129 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 581/1169] 10.222.0.42 () {60 vars in 1138 bytes} [Mon Sep 12 14:02:10 2022] POST /identity/v3/domains => generated 267 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:02:12.342621 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe3e92-2102-43b2-ba58-fbd64a0d04cf None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:12.343204 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe3e92-2102-43b2-ba58-fbd64a0d04cf None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:12.343204 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe3e92-2102-43b2-ba58-fbd64a0d04cf None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:12.344575 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 589/1170] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:02:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:02:12.350246 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b723b-4284-4967-b870-603e2e5744b2 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:12.350627 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b723b-4284-4967-b870-603e2e5744b2 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:12.350784 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b723b-4284-4967-b870-603e2e5744b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:12.368447 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-d77b723b-4284-4967-b870-603e2e5744b2 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:02:12.369614 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-d77b723b-4284-4967-b870-603e2e5744b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:02:12.433570 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 582/1171] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:02:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:02:12.696761 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe3e92-2102-43b2-ba58-fbd64a0d04cf None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:02:12.750422 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:02:12.797573 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe3e92-2102-43b2-ba58-fbd64a0d04cf None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:02:12.831610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc8481-d8dd-4279-a22e-3a48c666196b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:02:12.832246 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc8481-d8dd-4279-a22e-3a48c666196b None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:12.832549 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc8481-d8dd-4279-a22e-3a48c666196b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:12.832823 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc8481-d8dd-4279-a22e-3a48c666196b None admin] PATH_INFO: `/v3/domains/dd58dcaa8e634562ab12177f821f4370` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:12.842951 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc8481-d8dd-4279-a22e-3a48c666196b None admin] RBAC: Authorizing `identity:get_domain(domain_id=dd58dcaa8e634562ab12177f821f4370)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:02:12.844373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc8481-d8dd-4279-a22e-3a48c666196b None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:02:12.845943 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 590/1172] 10.222.0.42 () {56 vars in 1183 bytes} [Mon Sep 12 14:02:12 2022] GET /identity/v3/domains/dd58dcaa8e634562ab12177f821f4370 => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:02:12.851804 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b723b-4284-4967-b870-603e2e5744b2 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:02:12.861606 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:02:12.948651 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b723b-4284-4967-b870-603e2e5744b2 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:02:12.980018 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb20428-7c80-43cc-a58c-56662b0028c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:02:12.980606 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb20428-7c80-43cc-a58c-56662b0028c0 None admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:12.980879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb20428-7c80-43cc-a58c-56662b0028c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:12.981168 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb20428-7c80-43cc-a58c-56662b0028c0 None admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:12.981799 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb20428-7c80-43cc-a58c-56662b0028c0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:02:12.983225 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb20428-7c80-43cc-a58c-56662b0028c0 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:02:13.013478 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 583/1173] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Sep 12 14:02:12 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:02:15.153214 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96f296-e5a7-412a-ab89-82d664928222 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:15.153714 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96f296-e5a7-412a-ab89-82d664928222 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:15.153714 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96f296-e5a7-412a-ab89-82d664928222 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:15.155045 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 591/1174] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:02:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:02:15.160296 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c3157-1094-4967-a56d-290f534d219e None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:15.161329 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c3157-1094-4967-a56d-290f534d219e None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:15.161432 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c3157-1094-4967-a56d-290f534d219e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:15.179109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-fd9c3157-1094-4967-a56d-290f534d219e None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:02:15.180282 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-fd9c3157-1094-4967-a56d-290f534d219e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:02:15.233978 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:02:15.245783 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 584/1175] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:02:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:02:15.519051 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c96f296-e5a7-412a-ab89-82d664928222 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:02:15.616141 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c96f296-e5a7-412a-ab89-82d664928222 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:02:15.628386 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:02:15.649538 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32537c6-c922-4d1e-9571-573fb4926013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:02:15.650146 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32537c6-c922-4d1e-9571-573fb4926013 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:15.650420 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32537c6-c922-4d1e-9571-573fb4926013 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:15.650687 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32537c6-c922-4d1e-9571-573fb4926013 None admin] PATH_INFO: `/v3/domains/dd58dcaa8e634562ab12177f821f4370` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:15.651797 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32537c6-c922-4d1e-9571-573fb4926013 None admin] RBAC: Authorizing `identity:get_domain(domain_id=dd58dcaa8e634562ab12177f821f4370)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:02:15.653189 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32537c6-c922-4d1e-9571-573fb4926013 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:02:15.654702 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 592/1176] 10.222.0.42 () {56 vars in 1183 bytes} [Mon Sep 12 14:02:15 2022] GET /identity/v3/domains/dd58dcaa8e634562ab12177f821f4370 => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:02:15.660325 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c3157-1094-4967-a56d-290f534d219e None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:02:15.754884 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c3157-1094-4967-a56d-290f534d219e None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:02:15.788010 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d968d-db47-4ce0-bc5e-8b1cfa039cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:02:15.788535 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d968d-db47-4ce0-bc5e-8b1cfa039cb0 None admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:15.788811 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d968d-db47-4ce0-bc5e-8b1cfa039cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:15.789076 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d968d-db47-4ce0-bc5e-8b1cfa039cb0 None admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:15.789708 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0d968d-db47-4ce0-bc5e-8b1cfa039cb0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:02:15.791211 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0d968d-db47-4ce0-bc5e-8b1cfa039cb0 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:02:15.811825 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 585/1177] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Sep 12 14:02:15 2022] POST /identity/v3/users => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:02:17.971981 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e751a47-abdb-4658-8451-e2dfdcc79a22 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:17.972504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e751a47-abdb-4658-8451-e2dfdcc79a22 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:17.972504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e751a47-abdb-4658-8451-e2dfdcc79a22 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:17.974432 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 593/1178] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:02:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:02:17.979728 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3da551-6f86-4eea-998a-9146abc3e571 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:17.980119 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3da551-6f86-4eea-998a-9146abc3e571 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:17.980274 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3da551-6f86-4eea-998a-9146abc3e571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:17.998400 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-aa3da551-6f86-4eea-998a-9146abc3e571 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:02:17.999606 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-aa3da551-6f86-4eea-998a-9146abc3e571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:02:18.040307 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:02:18.064761 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 586/1179] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:02:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:02:18.333594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e751a47-abdb-4658-8451-e2dfdcc79a22 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:02:18.430293 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e751a47-abdb-4658-8451-e2dfdcc79a22 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:02:18.463059 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af97b3-19ec-4c0d-8adb-6b17b2de4c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:02:18.463670 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af97b3-19ec-4c0d-8adb-6b17b2de4c8a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:18.463948 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af97b3-19ec-4c0d-8adb-6b17b2de4c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:18.464218 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af97b3-19ec-4c0d-8adb-6b17b2de4c8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:18.477350 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af97b3-19ec-4c0d-8adb-6b17b2de4c8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:02:18.483935 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-f9af97b3-19ec-4c0d-8adb-6b17b2de4c8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:02:18.483935 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:02:18.483935 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:02:18.483935 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:02:18.483935 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:02:18.483935 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:02:18.483935 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:02:18.483935 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:02:18.483935 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:02:18.483935 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:02:18.483935 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:02:18.483935 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:02:18.483935 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:02:18.483935 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:02:18.483935 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:02:18.483935 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:02:18.485172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:02:18.485172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:02:18.485172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:02:18.485172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:02:18.485172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:02:18.485172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:02:18.485172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:02:18.485172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:02:18.485172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:02:18.485172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:02:18.485172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:02:18.485172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:02:18.485172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:02:18.485172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:02:18.485172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:02:18.485172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:02:18.486123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:02:18.486123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:02:18.486123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:02:18.486123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:02:18.486123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:02:18.486123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:02:18.486123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:02:18.486123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:02:18.486123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:02:18.486123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:02:18.486123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:02:18.486123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:02:18.486123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 594/1180] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 12 14:02:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:02:18.492551 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3da551-6f86-4eea-998a-9146abc3e571 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:02:18.588821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3da551-6f86-4eea-998a-9146abc3e571 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:02:18.620768 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860cda7f-3e9a-49ee-92bf-465fdfe0130c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:02:18.621364 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cda7f-3e9a-49ee-92bf-465fdfe0130c None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:18.621643 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cda7f-3e9a-49ee-92bf-465fdfe0130c None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:18.621915 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cda7f-3e9a-49ee-92bf-465fdfe0130c None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:18.622573 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860cda7f-3e9a-49ee-92bf-465fdfe0130c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:02:18.622900 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860cda7f-3e9a-49ee-92bf-465fdfe0130c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:02:18.624253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860cda7f-3e9a-49ee-92bf-465fdfe0130c None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:02:18.632473 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 587/1181] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 14:02:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:02:18.638316 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af97b3-19ec-4c0d-8adb-6b17b2de4c8a None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:02:18.679080 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:02:18.734695 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af97b3-19ec-4c0d-8adb-6b17b2de4c8a None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:02:18.772659 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389ba890-1c99-40e6-b119-75a1f4b3a84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:02:18.773283 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ba890-1c99-40e6-b119-75a1f4b3a84e None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:18.773564 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ba890-1c99-40e6-b119-75a1f4b3a84e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:18.773835 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ba890-1c99-40e6-b119-75a1f4b3a84e None admin] PATH_INFO: `/v3/users/ec717c03afbe447e96995071d28fd328` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:18.786485 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389ba890-1c99-40e6-b119-75a1f4b3a84e None admin] RBAC: Authorizing `identity:get_user(user_id=ec717c03afbe447e96995071d28fd328)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:02:18.787927 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389ba890-1c99-40e6-b119-75a1f4b3a84e None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:02:18.789501 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 595/1182] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 14:02:18 2022] GET /identity/v3/users/ec717c03afbe447e96995071d28fd328 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:02:18.795319 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860cda7f-3e9a-49ee-92bf-465fdfe0130c None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:02:18.837283 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:02:18.901594 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860cda7f-3e9a-49ee-92bf-465fdfe0130c None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:02:18.933575 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ee95c-931e-4988-ad43-f09bacb88bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:02:18.934167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ee95c-931e-4988-ad43-f09bacb88bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:18.934444 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ee95c-931e-4988-ad43-f09bacb88bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:18.934712 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ee95c-931e-4988-ad43-f09bacb88bf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:18.998551 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26ee95c-931e-4988-ad43-f09bacb88bf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:02:19.000008 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26ee95c-931e-4988-ad43-f09bacb88bf3 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:02:19.035959 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:02:19.094033 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 588/1183] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:02:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 14:02:19.100000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389ba890-1c99-40e6-b119-75a1f4b3a84e None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:02:19.192043 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:02:19.196543 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389ba890-1c99-40e6-b119-75a1f4b3a84e None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:02:19.229038 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab896f5-4e71-457c-883d-5f586c0af002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:02:19.229649 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab896f5-4e71-457c-883d-5f586c0af002 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:19.229925 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab896f5-4e71-457c-883d-5f586c0af002 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:19.230195 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab896f5-4e71-457c-883d-5f586c0af002 None admin] PATH_INFO: `/v3/projects/3bd915e83cbe40868c616988f5cbd241` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:19.231310 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab896f5-4e71-457c-883d-5f586c0af002 None admin] RBAC: Authorizing `identity:get_project(project_id=3bd915e83cbe40868c616988f5cbd241)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:02:19.232725 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab896f5-4e71-457c-883d-5f586c0af002 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:02:19.234321 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 596/1184] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 14:02:19 2022] GET /identity/v3/projects/3bd915e83cbe40868c616988f5cbd241 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:02:19.241137 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ee95c-931e-4988-ad43-f09bacb88bf3 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:02:19.336302 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ee95c-931e-4988-ad43-f09bacb88bf3 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:02:19.369055 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe5ab1-7aa8-4c53-b4ce-3b6a3007cf4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:02:19.369637 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe5ab1-7aa8-4c53-b4ce-3b6a3007cf4b None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:19.369910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe5ab1-7aa8-4c53-b4ce-3b6a3007cf4b None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:19.370173 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe5ab1-7aa8-4c53-b4ce-3b6a3007cf4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:19.370899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe5ab1-7aa8-4c53-b4ce-3b6a3007cf4b None admin] RBAC: Adding query filter params (role.id=e14564f00daf48a28d4e32da8cd2efad, scope.project.id=3bd915e83cbe40868c616988f5cbd241, user.id=ec717c03afbe447e96995071d28fd328) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:02:19.371199 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe5ab1-7aa8-4c53-b4ce-3b6a3007cf4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:02:19.372591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe5ab1-7aa8-4c53-b4ce-3b6a3007cf4b None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:02:19.377760 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 589/1185] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 14:02:19 2022] GET /identity/v3/role_assignments?user.id=ec717c03afbe447e96995071d28fd328&scope.project.id=3bd915e83cbe40868c616988f5cbd241&role.id=e14564f00daf48a28d4e32da8cd2efad => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:02:21.461865 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e7c76-3f76-4322-89f5-fd71de600131 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:21.462400 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e7c76-3f76-4322-89f5-fd71de600131 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:21.462400 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e7c76-3f76-4322-89f5-fd71de600131 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:21.463788 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 597/1186] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:02:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:02:21.468824 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842101d-1ccc-41ba-880f-f30ed2001b40 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:21.469263 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842101d-1ccc-41ba-880f-f30ed2001b40 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:21.469369 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842101d-1ccc-41ba-880f-f30ed2001b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:21.476061 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:02:21.489827 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-2842101d-1ccc-41ba-880f-f30ed2001b40 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:02:21.491022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-2842101d-1ccc-41ba-880f-f30ed2001b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:02:21.554126 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 590/1187] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:02:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:02:21.829165 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e7c76-3f76-4322-89f5-fd71de600131 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:02:21.927249 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e7c76-3f76-4322-89f5-fd71de600131 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:02:21.959772 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1ad14-851b-4983-98c3-5972214b812a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:02:21.960379 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1ad14-851b-4983-98c3-5972214b812a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:21.960657 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1ad14-851b-4983-98c3-5972214b812a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:21.960928 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1ad14-851b-4983-98c3-5972214b812a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:21.970667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1ad14-851b-4983-98c3-5972214b812a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:02:21.976218 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-f2a1ad14-851b-4983-98c3-5972214b812a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:02:21.976218 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:02:21.976218 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:02:21.976218 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:02:21.976218 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:02:21.976218 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:02:21.976218 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:02:21.976218 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:02:21.976218 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:02:21.976218 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:02:21.976218 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:02:21.976218 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:02:21.976218 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:02:21.976218 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:02:21.976218 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:02:21.976218 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:02:21.977299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:02:21.977299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:02:21.977299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:02:21.977299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:02:21.977299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:02:21.977299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:02:21.977299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:02:21.977299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:02:21.977299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:02:21.977299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:02:21.977299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:02:21.977299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:02:21.977299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:02:21.977299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:02:21.977299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:02:21.977299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:02:21.978305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:02:21.978305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:02:21.978305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:02:21.978305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:02:21.978305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:02:21.978305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:02:21.978305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:02:21.978305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:02:21.978305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:02:21.978305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:02:21.978305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:02:21.978305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:02:21.978305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 598/1188] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 12 14:02:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:02:21.983270 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842101d-1ccc-41ba-880f-f30ed2001b40 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:02:22.080395 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842101d-1ccc-41ba-880f-f30ed2001b40 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:02:22.110325 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:02:22.115582 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198ca1c-5cbd-4104-843a-92e7229645cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:02:22.116181 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198ca1c-5cbd-4104-843a-92e7229645cb None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:22.116462 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198ca1c-5cbd-4104-843a-92e7229645cb None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:22.116735 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198ca1c-5cbd-4104-843a-92e7229645cb None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:22.117397 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7198ca1c-5cbd-4104-843a-92e7229645cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:02:22.117698 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7198ca1c-5cbd-4104-843a-92e7229645cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:02:22.119095 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7198ca1c-5cbd-4104-843a-92e7229645cb None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:02:22.126988 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 591/1189] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 14:02:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:02:22.132970 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1ad14-851b-4983-98c3-5972214b812a None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:02:22.159858 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:02:22.229379 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1ad14-851b-4983-98c3-5972214b812a None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:02:22.261472 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff742b2-3a87-4a1f-a724-7c0db470fb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:02:22.262075 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff742b2-3a87-4a1f-a724-7c0db470fb0d None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:22.262348 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff742b2-3a87-4a1f-a724-7c0db470fb0d None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:22.262613 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff742b2-3a87-4a1f-a724-7c0db470fb0d None admin] PATH_INFO: `/v3/users/ec717c03afbe447e96995071d28fd328` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:22.263726 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff742b2-3a87-4a1f-a724-7c0db470fb0d None admin] RBAC: Authorizing `identity:get_user(user_id=ec717c03afbe447e96995071d28fd328)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:02:22.265119 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff742b2-3a87-4a1f-a724-7c0db470fb0d None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:02:22.266647 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 599/1190] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 14:02:22 2022] GET /identity/v3/users/ec717c03afbe447e96995071d28fd328 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:02:22.272879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198ca1c-5cbd-4104-843a-92e7229645cb None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:02:22.368139 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198ca1c-5cbd-4104-843a-92e7229645cb None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:02:22.400401 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c264e87a-0f40-4ece-8e12-6ff345f5e78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:02:22.401009 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264e87a-0f40-4ece-8e12-6ff345f5e78d None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:22.401285 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264e87a-0f40-4ece-8e12-6ff345f5e78d None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:22.401555 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264e87a-0f40-4ece-8e12-6ff345f5e78d None admin] PATH_INFO: `/v3/projects/3bd915e83cbe40868c616988f5cbd241` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:22.402656 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c264e87a-0f40-4ece-8e12-6ff345f5e78d None admin] RBAC: Authorizing `identity:get_project(project_id=3bd915e83cbe40868c616988f5cbd241)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:02:22.404088 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c264e87a-0f40-4ece-8e12-6ff345f5e78d None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:02:22.405679 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 592/1191] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 14:02:22 2022] GET /identity/v3/projects/3bd915e83cbe40868c616988f5cbd241 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:02:22.411645 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff742b2-3a87-4a1f-a724-7c0db470fb0d None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:02:22.499712 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:02:22.507145 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff742b2-3a87-4a1f-a724-7c0db470fb0d None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:02:22.539444 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1584d123-5dec-4093-9504-b695b2df2138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:02:22.540082 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584d123-5dec-4093-9504-b695b2df2138 None admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:22.540354 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584d123-5dec-4093-9504-b695b2df2138 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:22.540619 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584d123-5dec-4093-9504-b695b2df2138 None admin] PATH_INFO: `/v3/projects/3bd915e83cbe40868c616988f5cbd241/users/ec717c03afbe447e96995071d28fd328/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:22.542458 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1584d123-5dec-4093-9504-b695b2df2138 None admin] RBAC: Authorizing `identity:create_grant(project_id=3bd915e83cbe40868c616988f5cbd241, user_id=ec717c03afbe447e96995071d28fd328, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:02:22.543907 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1584d123-5dec-4093-9504-b695b2df2138 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:02:22.556123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 600/1192] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 12 14:02:22 2022] PUT /identity/v3/projects/3bd915e83cbe40868c616988f5cbd241/users/ec717c03afbe447e96995071d28fd328/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:02:24.656594 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c94ed-514e-4f0f-9c5c-29445616482c None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:24.657141 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c94ed-514e-4f0f-9c5c-29445616482c None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:24.657141 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c94ed-514e-4f0f-9c5c-29445616482c None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:24.658470 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 593/1193] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:02:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:02:24.663704 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376a1ac-ff3f-41c2-9ec7-190596fd50b1 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:24.664069 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376a1ac-ff3f-41c2-9ec7-190596fd50b1 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:24.664251 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376a1ac-ff3f-41c2-9ec7-190596fd50b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:24.681869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-9376a1ac-ff3f-41c2-9ec7-190596fd50b1 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:02:24.683078 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-9376a1ac-ff3f-41c2-9ec7-190596fd50b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:02:24.747089 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 601/1194] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:02:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:02:25.022150 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c94ed-514e-4f0f-9c5c-29445616482c None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:02:25.083054 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:02:25.130116 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c94ed-514e-4f0f-9c5c-29445616482c None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:02:25.161910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b90ab-f25c-4893-8584-9b7794ca3bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:02:25.162524 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b90ab-f25c-4893-8584-9b7794ca3bb1 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:25.162803 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b90ab-f25c-4893-8584-9b7794ca3bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:25.163102 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b90ab-f25c-4893-8584-9b7794ca3bb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:25.173278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720b90ab-f25c-4893-8584-9b7794ca3bb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:02:25.178732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-720b90ab-f25c-4893-8584-9b7794ca3bb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:02:25.178732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:02:25.178732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:02:25.178732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:02:25.178732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:02:25.178732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:02:25.178732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:02:25.178732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:02:25.178732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:02:25.178732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:02:25.178732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:02:25.178732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:02:25.178732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:02:25.178732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:02:25.178732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:02:25.178732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:02:25.179675 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:02:25.179675 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:02:25.179675 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:02:25.179675 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:02:25.179675 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:02:25.179675 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:02:25.179675 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:02:25.179675 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:02:25.179675 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:02:25.179675 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:02:25.179675 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:02:25.179675 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:02:25.179675 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:02:25.179675 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:02:25.179675 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:02:25.179675 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:02:25.180611 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:02:25.180611 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:02:25.180611 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:02:25.180611 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:02:25.180611 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:02:25.180611 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:02:25.180611 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:02:25.180611 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:02:25.180611 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:02:25.180611 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:02:25.180611 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:02:25.180611 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:02:25.180611 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 594/1195] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 12 14:02:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:02:25.185917 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9376a1ac-ff3f-41c2-9ec7-190596fd50b1 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:02:25.264940 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:02:25.280594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9376a1ac-ff3f-41c2-9ec7-190596fd50b1 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:02:25.313346 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56206ea1-d2a7-40ce-b5d9-ade6ffba762c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:02:25.313938 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56206ea1-d2a7-40ce-b5d9-ade6ffba762c None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:25.314217 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56206ea1-d2a7-40ce-b5d9-ade6ffba762c None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:25.314485 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56206ea1-d2a7-40ce-b5d9-ade6ffba762c None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:25.315168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56206ea1-d2a7-40ce-b5d9-ade6ffba762c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:02:25.315470 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56206ea1-d2a7-40ce-b5d9-ade6ffba762c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:02:25.316852 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56206ea1-d2a7-40ce-b5d9-ade6ffba762c None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:02:25.324761 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 602/1196] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 14:02:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:02:25.330491 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b90ab-f25c-4893-8584-9b7794ca3bb1 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:02:25.427526 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b90ab-f25c-4893-8584-9b7794ca3bb1 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:02:25.451292 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:02:25.462792 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf711be-75e3-4fa1-b5cb-34d4f901913e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:02:25.463443 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf711be-75e3-4fa1-b5cb-34d4f901913e None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:25.463720 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf711be-75e3-4fa1-b5cb-34d4f901913e None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:25.463989 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf711be-75e3-4fa1-b5cb-34d4f901913e None admin] PATH_INFO: `/v3/users/ec717c03afbe447e96995071d28fd328` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:25.465083 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf711be-75e3-4fa1-b5cb-34d4f901913e None admin] RBAC: Authorizing `identity:get_user(user_id=ec717c03afbe447e96995071d28fd328)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:02:25.466489 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf711be-75e3-4fa1-b5cb-34d4f901913e None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:02:25.468076 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 595/1197] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 12 14:02:25 2022] GET /identity/v3/users/ec717c03afbe447e96995071d28fd328 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:02:25.474355 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56206ea1-d2a7-40ce-b5d9-ade6ffba762c None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:02:25.570073 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56206ea1-d2a7-40ce-b5d9-ade6ffba762c None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:02:25.602396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851b0d5-3bce-4a3b-ad89-f023f32ddef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:02:25.603015 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851b0d5-3bce-4a3b-ad89-f023f32ddef1 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:25.603293 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851b0d5-3bce-4a3b-ad89-f023f32ddef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:25.603564 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851b0d5-3bce-4a3b-ad89-f023f32ddef1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:25.617747 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:02:25.667671 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b851b0d5-3bce-4a3b-ad89-f023f32ddef1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:02:25.669082 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b851b0d5-3bce-4a3b-ad89-f023f32ddef1 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:02:25.767064 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 603/1198] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:02:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 14:02:25.773523 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf711be-75e3-4fa1-b5cb-34d4f901913e None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:02:25.870555 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf711be-75e3-4fa1-b5cb-34d4f901913e None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:02:25.902441 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41eecd-338a-4401-9609-3b02fe8d6634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:02:25.903071 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41eecd-338a-4401-9609-3b02fe8d6634 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:25.903344 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41eecd-338a-4401-9609-3b02fe8d6634 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:25.903610 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41eecd-338a-4401-9609-3b02fe8d6634 None admin] PATH_INFO: `/v3/projects/3bd915e83cbe40868c616988f5cbd241` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:25.904692 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41eecd-338a-4401-9609-3b02fe8d6634 None admin] RBAC: Authorizing `identity:get_project(project_id=3bd915e83cbe40868c616988f5cbd241)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:02:25.906090 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41eecd-338a-4401-9609-3b02fe8d6634 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:02:25.907703 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 596/1199] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 12 14:02:25 2022] GET /identity/v3/projects/3bd915e83cbe40868c616988f5cbd241 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:02:25.914327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851b0d5-3bce-4a3b-ad89-f023f32ddef1 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:02:25.959178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:02:26.009769 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851b0d5-3bce-4a3b-ad89-f023f32ddef1 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:02:26.042336 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c90fb22-214a-48bf-8aad-c8aeb121c500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:02:26.042955 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90fb22-214a-48bf-8aad-c8aeb121c500 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:02:26.043234 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90fb22-214a-48bf-8aad-c8aeb121c500 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:02:26.043507 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90fb22-214a-48bf-8aad-c8aeb121c500 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:02:26.044222 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c90fb22-214a-48bf-8aad-c8aeb121c500 None admin] RBAC: Adding query filter params (role.id=e14564f00daf48a28d4e32da8cd2efad, scope.project.id=3bd915e83cbe40868c616988f5cbd241, user.id=ec717c03afbe447e96995071d28fd328) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:02:26.044527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c90fb22-214a-48bf-8aad-c8aeb121c500 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:02:26.045924 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c90fb22-214a-48bf-8aad-c8aeb121c500 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:02:26.051556 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 604/1200] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 14:02:25 2022] GET /identity/v3/role_assignments?user.id=ec717c03afbe447e96995071d28fd328&scope.project.id=3bd915e83cbe40868c616988f5cbd241&role.id=e14564f00daf48a28d4e32da8cd2efad => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:02:35.898432 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=71 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:02:36.048936 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=45 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:21.668405 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05645b-6163-4b6a-8ece-05ea3d59f519 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:21.669040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05645b-6163-4b6a-8ece-05ea3d59f519 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:21.669040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05645b-6163-4b6a-8ece-05ea3d59f519 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:21.670427 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 597/1201] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:03:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:03:21.677045 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1308682-0bac-4187-8226-7d2b79df3a42 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:21.677586 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1308682-0bac-4187-8226-7d2b79df3a42 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:21.677586 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1308682-0bac-4187-8226-7d2b79df3a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:21.702994 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-e1308682-0bac-4187-8226-7d2b79df3a42 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:03:21.704370 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e1308682-0bac-4187-8226-7d2b79df3a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:03:21.800372 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 605/1202] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:03:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:03:22.092075 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd05645b-6163-4b6a-8ece-05ea3d59f519 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:22.243032 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd05645b-6163-4b6a-8ece-05ea3d59f519 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:22.286298 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2767902-af53-4397-ae3d-1266c1bfb761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:22.287332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2767902-af53-4397-ae3d-1266c1bfb761 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:22.287715 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2767902-af53-4397-ae3d-1266c1bfb761 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:22.288038 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2767902-af53-4397-ae3d-1266c1bfb761 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:22.305635 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2767902-af53-4397-ae3d-1266c1bfb761 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:22.307838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2767902-af53-4397-ae3d-1266c1bfb761 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:22.327881 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-b2767902-af53-4397-ae3d-1266c1bfb761 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:03:22.327881 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:03:22.327881 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:03:22.327881 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:03:22.327881 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:03:22.327881 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:03:22.327881 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:03:22.327881 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:03:22.327881 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:03:22.327881 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:03:22.327881 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:03:22.327881 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:03:22.327881 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:03:22.327881 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:03:22.327881 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:03:22.327881 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:03:22.329023 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:03:22.329023 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:03:22.329023 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:03:22.329023 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:03:22.329023 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:03:22.329023 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:03:22.329023 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:03:22.329023 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:03:22.329023 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:03:22.329023 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:03:22.329023 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:03:22.329023 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:03:22.329023 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:03:22.329023 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:03:22.329023 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:03:22.329023 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:03:22.329023 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:03:22.329991 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:03:22.329991 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:03:22.329991 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:03:22.329991 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 598/1203] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:03:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:03:22.335904 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1308682-0bac-4187-8226-7d2b79df3a42 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:22.472343 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1308682-0bac-4187-8226-7d2b79df3a42 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:22.521557 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:22.530589 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e74a8-cadf-403a-af1e-37c1e0126481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:22.531446 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e74a8-cadf-403a-af1e-37c1e0126481 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:22.531728 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e74a8-cadf-403a-af1e-37c1e0126481 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:22.532003 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e74a8-cadf-403a-af1e-37c1e0126481 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:22.532755 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7e74a8-cadf-403a-af1e-37c1e0126481 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:22.533059 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7e74a8-cadf-403a-af1e-37c1e0126481 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:22.535435 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7e74a8-cadf-403a-af1e-37c1e0126481 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:22.550098 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 606/1204] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:03:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:03:22.556697 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2767902-af53-4397-ae3d-1266c1bfb761 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:22.628765 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:22.686683 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2767902-af53-4397-ae3d-1266c1bfb761 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:22.733260 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20bc7c6-7f44-4e37-a6fb-1ba9c1c43a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:22.733767 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20bc7c6-7f44-4e37-a6fb-1ba9c1c43a3f None admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:22.734060 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20bc7c6-7f44-4e37-a6fb-1ba9c1c43a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:22.734287 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20bc7c6-7f44-4e37-a6fb-1ba9c1c43a3f None admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:22.735799 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20bc7c6-7f44-4e37-a6fb-1ba9c1c43a3f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:22.738500 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20bc7c6-7f44-4e37-a6fb-1ba9c1c43a3f None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:22.762422 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 599/1205] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Sep 12 14:03:22 2022] POST /identity/v3/users => generated 251 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:03:25.264654 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593232b-661a-439c-95d3-5cd4fbde9a03 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:25.265185 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593232b-661a-439c-95d3-5cd4fbde9a03 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:25.265185 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593232b-661a-439c-95d3-5cd4fbde9a03 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:25.266573 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 607/1206] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:03:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:03:25.273499 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2c1f4-2f54-4790-be4f-e766269ad06c None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:25.273970 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2c1f4-2f54-4790-be4f-e766269ad06c None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:25.274086 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2c1f4-2f54-4790-be4f-e766269ad06c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:25.296472 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-39d2c1f4-2f54-4790-be4f-e766269ad06c None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:03:25.298017 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-39d2c1f4-2f54-4790-be4f-e766269ad06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:03:25.380333 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 600/1207] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:03:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:03:25.708295 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9593232b-661a-439c-95d3-5cd4fbde9a03 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:25.842254 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9593232b-661a-439c-95d3-5cd4fbde9a03 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:25.882531 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2dd8a3-c236-4e5d-b58b-8cb92e26f547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:25.883228 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2dd8a3-c236-4e5d-b58b-8cb92e26f547 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:25.883567 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2dd8a3-c236-4e5d-b58b-8cb92e26f547 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:25.883867 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2dd8a3-c236-4e5d-b58b-8cb92e26f547 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:25.895745 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2dd8a3-c236-4e5d-b58b-8cb92e26f547 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:03:25.900781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-9b2dd8a3-c236-4e5d-b58b-8cb92e26f547 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:03:25.900781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:03:25.900781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:03:25.900781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:03:25.900781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:03:25.900781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:03:25.900781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:03:25.900781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:03:25.900781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:03:25.900781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:03:25.900781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:03:25.900781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:03:25.900781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:03:25.900781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:03:25.900781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:03:25.900781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:03:25.902032 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:03:25.902032 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:03:25.902032 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:03:25.902032 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:03:25.902032 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:03:25.902032 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:03:25.902032 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:03:25.902032 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:03:25.902032 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:03:25.902032 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:03:25.902032 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:03:25.902032 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:03:25.902032 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:03:25.902032 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:03:25.902032 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:03:25.902032 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:03:25.902904 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:03:25.902904 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:03:25.902904 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:03:25.902904 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:03:25.902904 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:03:25.902904 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:03:25.902904 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:03:25.902904 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:03:25.902904 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:03:25.902904 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:03:25.902904 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:03:25.902904 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:03:25.902904 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 608/1208] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 12 14:03:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:03:25.910009 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2c1f4-2f54-4790-be4f-e766269ad06c None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:25.953474 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:26.031353 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2c1f4-2f54-4790-be4f-e766269ad06c None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:26.076960 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67989839-fed5-435d-9174-ec9a72ae59f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:26.077585 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67989839-fed5-435d-9174-ec9a72ae59f5 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:26.077856 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67989839-fed5-435d-9174-ec9a72ae59f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:26.078116 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67989839-fed5-435d-9174-ec9a72ae59f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:26.079712 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67989839-fed5-435d-9174-ec9a72ae59f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:26.079712 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67989839-fed5-435d-9174-ec9a72ae59f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:26.080524 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67989839-fed5-435d-9174-ec9a72ae59f5 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:26.089638 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 601/1209] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 12 14:03:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:03:26.096048 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2dd8a3-c236-4e5d-b58b-8cb92e26f547 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:26.154793 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:26.212645 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2dd8a3-c236-4e5d-b58b-8cb92e26f547 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:26.250302 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc58a84-3c0c-4907-bb52-f79cbd630d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:26.250946 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc58a84-3c0c-4907-bb52-f79cbd630d8c None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:26.251274 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc58a84-3c0c-4907-bb52-f79cbd630d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:26.251545 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc58a84-3c0c-4907-bb52-f79cbd630d8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:26.262820 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc58a84-3c0c-4907-bb52-f79cbd630d8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:26.264298 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc58a84-3c0c-4907-bb52-f79cbd630d8c None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:26.280524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-9dc58a84-3c0c-4907-bb52-f79cbd630d8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:03:26.280524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:03:26.280524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:03:26.280524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:03:26.280524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:03:26.280524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:03:26.280524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:03:26.280524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:03:26.280524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:03:26.280524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:03:26.280524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:03:26.280524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:03:26.280524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:03:26.280524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:03:26.280524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:03:26.280524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:03:26.282055 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:03:26.282055 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:03:26.282055 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:03:26.282055 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:03:26.282055 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:03:26.282055 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:03:26.282055 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:03:26.282055 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:03:26.282055 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:03:26.282055 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:03:26.282055 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:03:26.282055 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:03:26.282055 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:03:26.282055 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:03:26.282055 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:03:26.282055 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:03:26.282055 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:03:26.283005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:03:26.283005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:03:26.283005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:03:26.283005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 609/1210] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:03:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:03:26.288414 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67989839-fed5-435d-9174-ec9a72ae59f5 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:26.398760 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:26.402148 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67989839-fed5-435d-9174-ec9a72ae59f5 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:26.439749 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d35d9c-da18-4f74-897f-d43ae46e0e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:26.439749 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d35d9c-da18-4f74-897f-d43ae46e0e28 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:26.439749 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d35d9c-da18-4f74-897f-d43ae46e0e28 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:26.439749 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d35d9c-da18-4f74-897f-d43ae46e0e28 None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:26.442640 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d35d9c-da18-4f74-897f-d43ae46e0e28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:26.443468 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d35d9c-da18-4f74-897f-d43ae46e0e28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:26.445044 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d35d9c-da18-4f74-897f-d43ae46e0e28 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:26.462422 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 602/1211] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:03:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:03:26.468655 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc58a84-3c0c-4907-bb52-f79cbd630d8c None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:26.587984 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc58a84-3c0c-4907-bb52-f79cbd630d8c None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:26.589886 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:26.628633 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b92aa0-dbf3-44ad-a5ba-553145042599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:26.629320 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b92aa0-dbf3-44ad-a5ba-553145042599 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:26.629623 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b92aa0-dbf3-44ad-a5ba-553145042599 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:26.629909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b92aa0-dbf3-44ad-a5ba-553145042599 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:26.638553 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b92aa0-dbf3-44ad-a5ba-553145042599 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:26.640183 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b92aa0-dbf3-44ad-a5ba-553145042599 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:26.652447 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-89b92aa0-dbf3-44ad-a5ba-553145042599 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 12 14:03:26.652447 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:03:26.652447 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:03:26.652447 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:03:26.652447 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:03:26.652447 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:03:26.652447 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:03:26.652447 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:03:26.652447 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:03:26.652447 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:03:26.652447 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:03:26.652447 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:03:26.652447 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:03:26.652447 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:03:26.652447 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:03:26.652447 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:03:26.653524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:03:26.653524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:03:26.653524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:03:26.653524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:03:26.653524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:03:26.653524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:03:26.653524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:03:26.653524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:03:26.653524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:03:26.653524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:03:26.653524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:03:26.653524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:03:26.653524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:03:26.653524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:03:26.653524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:03:26.653524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:03:26.653524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:03:26.653524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:03:26.654522 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:03:26.654522 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:03:26.654522 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:03:26.654522 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:03:26.654522 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:03:26.654522 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:03:26.654522 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:03:26.654522 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:03:26.654522 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 12 14:03:26.654522 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:03:26.654522 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 610/1212] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:03:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:03:26.660068 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d35d9c-da18-4f74-897f-d43ae46e0e28 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:26.774487 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d35d9c-da18-4f74-897f-d43ae46e0e28 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:26.810230 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cce184-e72e-487d-8c4b-6694e8b9b19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:26.810847 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cce184-e72e-487d-8c4b-6694e8b9b19b None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:26.811139 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cce184-e72e-487d-8c4b-6694e8b9b19b None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:26.811401 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cce184-e72e-487d-8c4b-6694e8b9b19b None admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:26.812114 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cce184-e72e-487d-8c4b-6694e8b9b19b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:26.812416 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cce184-e72e-487d-8c4b-6694e8b9b19b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:26.814108 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cce184-e72e-487d-8c4b-6694e8b9b19b None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:26.838330 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 603/1213] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Sep 12 14:03:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:03:26.839322 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b92aa0-dbf3-44ad-a5ba-553145042599 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:26.953713 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b92aa0-dbf3-44ad-a5ba-553145042599 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:26.993720 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e7da6-a3f9-4261-b1b3-7c7468393c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:26.994500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e7da6-a3f9-4261-b1b3-7c7468393c32 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:26.994776 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e7da6-a3f9-4261-b1b3-7c7468393c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:26.995209 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e7da6-a3f9-4261-b1b3-7c7468393c32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:27.031337 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:27.089919 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47e7da6-a3f9-4261-b1b3-7c7468393c32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:27.091337 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47e7da6-a3f9-4261-b1b3-7c7468393c32 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:27.215180 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 611/1214] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:03:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 12 14:03:27.220957 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cce184-e72e-487d-8c4b-6694e8b9b19b None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:27.241445 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:27.344905 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cce184-e72e-487d-8c4b-6694e8b9b19b None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:27.394740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea484fd-212a-4d57-9340-627f15ef2b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:27.395468 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea484fd-212a-4d57-9340-627f15ef2b7a None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:27.395733 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea484fd-212a-4d57-9340-627f15ef2b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:27.396096 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea484fd-212a-4d57-9340-627f15ef2b7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:27.407698 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea484fd-212a-4d57-9340-627f15ef2b7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:27.409452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea484fd-212a-4d57-9340-627f15ef2b7a None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:27.427512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-eea484fd-212a-4d57-9340-627f15ef2b7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:03:27.427512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:03:27.427512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:03:27.427512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:03:27.427512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:03:27.427512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:03:27.427512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:03:27.427512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:03:27.427512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:03:27.427512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:03:27.427512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:03:27.427512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:03:27.427512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:03:27.427512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:03:27.427512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:03:27.427512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:03:27.428825 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:03:27.428825 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:03:27.428825 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:03:27.428825 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:03:27.428825 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:03:27.428825 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:03:27.428825 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:03:27.428825 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:03:27.428825 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:03:27.428825 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:03:27.428825 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:03:27.428825 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:03:27.428825 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:03:27.428825 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:03:27.428825 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:03:27.428825 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:03:27.428825 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:03:27.430045 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:03:27.430045 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:03:27.430045 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:03:27.430045 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 604/1215] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:03:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:03:27.434322 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e7da6-a3f9-4261-b1b3-7c7468393c32 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:27.506101 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:27.549057 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e7da6-a3f9-4261-b1b3-7c7468393c32 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:27.586781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4909622-d1a4-42e8-8fc1-efe0eb0c60ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:27.587410 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4909622-d1a4-42e8-8fc1-efe0eb0c60ec None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:27.587694 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4909622-d1a4-42e8-8fc1-efe0eb0c60ec None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:27.587966 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4909622-d1a4-42e8-8fc1-efe0eb0c60ec None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:27.588688 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4909622-d1a4-42e8-8fc1-efe0eb0c60ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:27.589043 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4909622-d1a4-42e8-8fc1-efe0eb0c60ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:27.590419 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4909622-d1a4-42e8-8fc1-efe0eb0c60ec None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:27.602490 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 612/1216] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:03:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:03:27.608743 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea484fd-212a-4d57-9340-627f15ef2b7a None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:27.691797 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:27.729908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea484fd-212a-4d57-9340-627f15ef2b7a None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:27.771761 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799228ad-e4d3-4133-84ae-019719367fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:27.772459 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799228ad-e4d3-4133-84ae-019719367fb5 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:27.772804 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799228ad-e4d3-4133-84ae-019719367fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:27.773079 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799228ad-e4d3-4133-84ae-019719367fb5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:27.780691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799228ad-e4d3-4133-84ae-019719367fb5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:27.782264 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799228ad-e4d3-4133-84ae-019719367fb5 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:27.794940 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-799228ad-e4d3-4133-84ae-019719367fb5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:03:27.794940 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:03:27.794940 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:03:27.794940 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:03:27.794940 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:03:27.794940 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:03:27.794940 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:03:27.794940 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:03:27.794940 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:03:27.794940 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:03:27.794940 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:03:27.794940 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:03:27.794940 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:03:27.794940 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:03:27.794940 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:03:27.796278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:03:27.796278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:03:27.796278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:03:27.796278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:03:27.796278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:03:27.796278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:03:27.796278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:03:27.796278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:03:27.796278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:03:27.796278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:03:27.796278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:03:27.796278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:03:27.796278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:03:27.796278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:03:27.796278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:03:27.796278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:03:27.796278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:03:27.797401 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:03:27.797401 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:03:27.797401 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:03:27.797401 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:03:27.797401 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:03:27.797401 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:03:27.797401 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:03:27.797401 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:03:27.797401 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 605/1217] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:03:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:03:27.804530 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4909622-d1a4-42e8-8fc1-efe0eb0c60ec None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:27.910214 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4909622-d1a4-42e8-8fc1-efe0eb0c60ec None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:27.936582 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:27.950827 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68521f58-99ab-4a10-8a25-bf4f73b345c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:27.951546 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68521f58-99ab-4a10-8a25-bf4f73b345c7 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:27.951901 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68521f58-99ab-4a10-8a25-bf4f73b345c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:27.952246 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68521f58-99ab-4a10-8a25-bf4f73b345c7 None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:27.953059 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68521f58-99ab-4a10-8a25-bf4f73b345c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:27.953432 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68521f58-99ab-4a10-8a25-bf4f73b345c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:27.955018 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68521f58-99ab-4a10-8a25-bf4f73b345c7 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:27.968103 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 613/1218] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 12 14:03:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:03:27.974163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799228ad-e4d3-4133-84ae-019719367fb5 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:28.085960 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799228ad-e4d3-4133-84ae-019719367fb5 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:28.114246 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:28.120607 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f883090-a1f9-4148-a1cb-b3ebc0fb53c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:28.121212 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f883090-a1f9-4148-a1cb-b3ebc0fb53c7 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:28.121493 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f883090-a1f9-4148-a1cb-b3ebc0fb53c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:28.121843 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f883090-a1f9-4148-a1cb-b3ebc0fb53c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:28.122486 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f883090-a1f9-4148-a1cb-b3ebc0fb53c7 None admin] RBAC: Adding query filter params (role.id=1c413f4e78ef45b7a821bb884f9cea07, scope.project.id=f666a617f8d14eb18bbeb076032b82f2, user.id=ce508bb9aa164a7fb1831a9b5ff5b932) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:28.122790 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f883090-a1f9-4148-a1cb-b3ebc0fb53c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:28.124236 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f883090-a1f9-4148-a1cb-b3ebc0fb53c7 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:28.130344 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 606/1219] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 14:03:27 2022] GET /identity/v3/role_assignments?user.id=ce508bb9aa164a7fb1831a9b5ff5b932&scope.project.id=f666a617f8d14eb18bbeb076032b82f2&role.id=1c413f4e78ef45b7a821bb884f9cea07 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:03:30.289873 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992838ac-45fb-45e8-8c28-f4d56a27ee51 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:30.290511 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992838ac-45fb-45e8-8c28-f4d56a27ee51 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:30.290511 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992838ac-45fb-45e8-8c28-f4d56a27ee51 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:30.291780 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 614/1220] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:03:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:03:30.296904 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e705fb-7f0d-4da4-952f-f37583fc38a6 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:30.297317 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e705fb-7f0d-4da4-952f-f37583fc38a6 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:30.297432 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e705fb-7f0d-4da4-952f-f37583fc38a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:30.316860 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-a1e705fb-7f0d-4da4-952f-f37583fc38a6 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:03:30.318067 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-a1e705fb-7f0d-4da4-952f-f37583fc38a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:03:30.389644 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 607/1221] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:03:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:03:30.678737 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992838ac-45fb-45e8-8c28-f4d56a27ee51 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:30.789092 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992838ac-45fb-45e8-8c28-f4d56a27ee51 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:30.829938 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09396196-145e-4bb8-be61-08284bf73a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:30.831362 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09396196-145e-4bb8-be61-08284bf73a85 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:30.831766 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09396196-145e-4bb8-be61-08284bf73a85 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:30.832139 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09396196-145e-4bb8-be61-08284bf73a85 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:30.843442 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09396196-145e-4bb8-be61-08284bf73a85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:03:30.848499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-09396196-145e-4bb8-be61-08284bf73a85 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:03:30.848499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:03:30.848499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:03:30.848499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:03:30.848499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:03:30.848499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:03:30.848499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:03:30.848499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:03:30.848499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:03:30.848499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:03:30.848499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:03:30.848499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:03:30.848499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:03:30.848499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:03:30.848499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:03:30.848499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:03:30.849506 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:03:30.849506 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:03:30.849506 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:03:30.849506 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:03:30.849506 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:03:30.849506 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:03:30.849506 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:03:30.849506 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:03:30.849506 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:03:30.849506 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:03:30.849506 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:03:30.849506 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:03:30.849506 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:03:30.849506 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:03:30.849506 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:03:30.849506 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:03:30.850356 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:03:30.850356 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:03:30.850356 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:03:30.850356 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:03:30.850356 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:03:30.850356 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:03:30.850356 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:03:30.850356 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:03:30.850356 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:03:30.850356 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:03:30.850356 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:03:30.850356 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:03:30.850356 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 615/1222] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 12 14:03:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:03:30.855641 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e705fb-7f0d-4da4-952f-f37583fc38a6 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:30.960786 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e705fb-7f0d-4da4-952f-f37583fc38a6 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:30.985285 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:30.997987 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08dee4e-6070-4814-897b-3c82e9018432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:30.998590 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08dee4e-6070-4814-897b-3c82e9018432 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:30.998888 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08dee4e-6070-4814-897b-3c82e9018432 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:30.999261 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08dee4e-6070-4814-897b-3c82e9018432 None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:30.999860 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08dee4e-6070-4814-897b-3c82e9018432 None admin] RBAC: Adding query filter params (name=service) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:31.000157 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08dee4e-6070-4814-897b-3c82e9018432 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:31.001541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08dee4e-6070-4814-897b-3c82e9018432 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:31.010103 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 608/1223] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 12 14:03:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:03:31.016384 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09396196-145e-4bb8-be61-08284bf73a85 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:31.061009 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:31.123890 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09396196-145e-4bb8-be61-08284bf73a85 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:31.159265 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057af804-f1e2-4400-ae84-69fcf80c77c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:31.159897 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057af804-f1e2-4400-ae84-69fcf80c77c7 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:31.160174 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057af804-f1e2-4400-ae84-69fcf80c77c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:31.160445 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057af804-f1e2-4400-ae84-69fcf80c77c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:31.179894 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057af804-f1e2-4400-ae84-69fcf80c77c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:31.182325 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057af804-f1e2-4400-ae84-69fcf80c77c7 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:31.200831 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-057af804-f1e2-4400-ae84-69fcf80c77c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:03:31.200831 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:03:31.200831 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:03:31.200831 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:03:31.200831 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:03:31.200831 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:03:31.200831 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:03:31.200831 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:03:31.200831 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:03:31.200831 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:03:31.200831 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:03:31.200831 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:03:31.200831 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:03:31.200831 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:03:31.200831 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:03:31.200831 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:03:31.201834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:03:31.201834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:03:31.201834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:03:31.201834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:03:31.201834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:03:31.201834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:03:31.201834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:03:31.201834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:03:31.201834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:03:31.201834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:03:31.201834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:03:31.201834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:03:31.201834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:03:31.201834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:03:31.201834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:03:31.201834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:03:31.201834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:03:31.207797 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:03:31.207797 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:03:31.207797 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:03:31.207797 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 616/1224] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:03:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:03:31.208611 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08dee4e-6070-4814-897b-3c82e9018432 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:31.327579 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08dee4e-6070-4814-897b-3c82e9018432 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:31.366980 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a8ae65-58f8-4aa9-9e38-b113a4edcc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:31.367595 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8ae65-58f8-4aa9-9e38-b113a4edcc8e None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:31.367861 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8ae65-58f8-4aa9-9e38-b113a4edcc8e None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:31.368126 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8ae65-58f8-4aa9-9e38-b113a4edcc8e None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:31.368758 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a8ae65-58f8-4aa9-9e38-b113a4edcc8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:31.369056 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a8ae65-58f8-4aa9-9e38-b113a4edcc8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:31.370403 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a8ae65-58f8-4aa9-9e38-b113a4edcc8e None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:31.382577 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 609/1225] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:03:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:03:31.389229 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057af804-f1e2-4400-ae84-69fcf80c77c7 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:31.492707 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:31.501862 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057af804-f1e2-4400-ae84-69fcf80c77c7 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:31.540945 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403e8e5-98ae-4a10-817c-366deffedcad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:31.541576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403e8e5-98ae-4a10-817c-366deffedcad None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:31.541857 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403e8e5-98ae-4a10-817c-366deffedcad None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:31.542128 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403e8e5-98ae-4a10-817c-366deffedcad None admin] PATH_INFO: `/v3/users/placement` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:31.550933 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403e8e5-98ae-4a10-817c-366deffedcad None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:31.552357 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403e8e5-98ae-4a10-817c-366deffedcad None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:31.563627 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-0403e8e5-98ae-4a10-817c-366deffedcad None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 12 14:03:31.563627 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:03:31.563627 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:03:31.563627 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:03:31.563627 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:03:31.563627 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:03:31.563627 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:03:31.563627 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:03:31.563627 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:03:31.563627 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:03:31.563627 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:03:31.563627 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:03:31.563627 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:03:31.563627 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:03:31.563627 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:03:31.563627 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:03:31.564730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:03:31.564730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:03:31.564730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:03:31.564730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:03:31.564730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:03:31.564730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:03:31.564730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:03:31.564730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:03:31.564730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:03:31.564730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:03:31.564730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:03:31.564730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:03:31.564730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:03:31.564730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:03:31.564730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:03:31.564730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:03:31.564730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:03:31.564730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:03:31.565689 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:03:31.565689 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:03:31.565689 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:03:31.565689 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:03:31.565689 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:03:31.565689 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:03:31.565689 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:03:31.565689 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:03:31.565689 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 12 14:03:31.565689 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:03:31.565689 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 617/1226] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:03:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:03:31.571593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a8ae65-58f8-4aa9-9e38-b113a4edcc8e None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:31.622021 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:31.677367 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a8ae65-58f8-4aa9-9e38-b113a4edcc8e None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:31.716349 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be31a5-2d95-46cf-af73-814e82bd9b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:31.716990 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be31a5-2d95-46cf-af73-814e82bd9b74 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:31.717278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be31a5-2d95-46cf-af73-814e82bd9b74 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:31.717541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be31a5-2d95-46cf-af73-814e82bd9b74 None admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:31.718245 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51be31a5-2d95-46cf-af73-814e82bd9b74 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:31.718605 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51be31a5-2d95-46cf-af73-814e82bd9b74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:31.720182 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51be31a5-2d95-46cf-af73-814e82bd9b74 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:31.741741 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 610/1227] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Sep 12 14:03:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:03:31.748146 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403e8e5-98ae-4a10-817c-366deffedcad None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:31.877978 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403e8e5-98ae-4a10-817c-366deffedcad None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:31.913330 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df03b35-6b26-4355-852f-7899d784a51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:31.914095 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df03b35-6b26-4355-852f-7899d784a51e None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:31.914355 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df03b35-6b26-4355-852f-7899d784a51e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:31.914747 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df03b35-6b26-4355-852f-7899d784a51e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:31.926516 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df03b35-6b26-4355-852f-7899d784a51e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:31.928420 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df03b35-6b26-4355-852f-7899d784a51e None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:31.943909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-9df03b35-6b26-4355-852f-7899d784a51e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:03:31.943909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:03:31.943909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:03:31.943909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:03:31.943909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:03:31.943909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:03:31.943909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:03:31.943909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:03:31.943909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:03:31.943909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:03:31.943909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:03:31.943909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:03:31.943909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:03:31.943909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:03:31.943909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:03:31.943909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:03:31.945672 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:03:31.945672 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:03:31.945672 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:03:31.945672 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:03:31.945672 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:03:31.945672 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:03:31.945672 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:03:31.945672 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:03:31.945672 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:03:31.945672 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:03:31.945672 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:03:31.945672 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:03:31.945672 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:03:31.945672 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:03:31.945672 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:03:31.945672 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:03:31.945672 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:03:31.946897 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:03:31.946897 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:03:31.946897 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:03:31.946897 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 618/1228] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:03:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:03:31.951263 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be31a5-2d95-46cf-af73-814e82bd9b74 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:32.057050 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be31a5-2d95-46cf-af73-814e82bd9b74 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:32.059685 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:32.093352 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbfefb2-ffbf-4aa2-8e18-c65808ed6c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:32.093994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbfefb2-ffbf-4aa2-8e18-c65808ed6c6e None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:32.094275 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbfefb2-ffbf-4aa2-8e18-c65808ed6c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:32.094598 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbfefb2-ffbf-4aa2-8e18-c65808ed6c6e None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:32.095341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbfefb2-ffbf-4aa2-8e18-c65808ed6c6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:32.095701 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbfefb2-ffbf-4aa2-8e18-c65808ed6c6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:32.097692 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbfefb2-ffbf-4aa2-8e18-c65808ed6c6e None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:32.115045 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 611/1229] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:03:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:03:32.116298 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df03b35-6b26-4355-852f-7899d784a51e None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:32.126018 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:32.224276 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df03b35-6b26-4355-852f-7899d784a51e None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:32.259327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a4b9a-b66a-4fea-9104-34f647d2cc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:32.259986 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a4b9a-b66a-4fea-9104-34f647d2cc0d None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:32.260268 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a4b9a-b66a-4fea-9104-34f647d2cc0d None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:32.260538 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a4b9a-b66a-4fea-9104-34f647d2cc0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:32.267644 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724a4b9a-b66a-4fea-9104-34f647d2cc0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:32.269334 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724a4b9a-b66a-4fea-9104-34f647d2cc0d None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:32.281609 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-724a4b9a-b66a-4fea-9104-34f647d2cc0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:03:32.281609 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:03:32.281609 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:03:32.281609 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:03:32.281609 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:03:32.281609 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:03:32.281609 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:03:32.281609 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:03:32.281609 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:03:32.281609 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:03:32.281609 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:03:32.281609 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:03:32.281609 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:03:32.281609 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:03:32.281609 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:03:32.282681 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:03:32.282681 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:03:32.282681 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:03:32.282681 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:03:32.282681 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:03:32.282681 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:03:32.282681 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:03:32.282681 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:03:32.282681 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:03:32.282681 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:03:32.282681 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:03:32.282681 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:03:32.282681 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:03:32.282681 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:03:32.282681 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:03:32.282681 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:03:32.282681 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:03:32.284387 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:03:32.284387 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:03:32.284387 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:03:32.284387 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:03:32.284387 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:03:32.284387 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:03:32.284387 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:03:32.284387 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:03:32.284387 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 619/1230] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:03:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:03:32.289848 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbfefb2-ffbf-4aa2-8e18-c65808ed6c6e None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:32.397979 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbfefb2-ffbf-4aa2-8e18-c65808ed6c6e None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:32.434086 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d04c65-9b2e-4572-95f3-d11ed7365ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:32.434717 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d04c65-9b2e-4572-95f3-d11ed7365ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:32.434993 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d04c65-9b2e-4572-95f3-d11ed7365ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:32.435268 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d04c65-9b2e-4572-95f3-d11ed7365ca4 None admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:32.435986 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d04c65-9b2e-4572-95f3-d11ed7365ca4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:32.436260 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d04c65-9b2e-4572-95f3-d11ed7365ca4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:32.437713 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d04c65-9b2e-4572-95f3-d11ed7365ca4 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:32.450400 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 612/1231] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 12 14:03:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:03:32.457390 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a4b9a-b66a-4fea-9104-34f647d2cc0d None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:32.528520 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:32.567405 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a4b9a-b66a-4fea-9104-34f647d2cc0d None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:32.604754 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d3d5be-3365-430f-9636-670f2a4c929e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:32.605495 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3d5be-3365-430f-9636-670f2a4c929e None admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:32.605792 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3d5be-3365-430f-9636-670f2a4c929e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:32.606188 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3d5be-3365-430f-9636-670f2a4c929e None admin] PATH_INFO: `/v3/projects/f666a617f8d14eb18bbeb076032b82f2/users/ce508bb9aa164a7fb1831a9b5ff5b932/roles/1c413f4e78ef45b7a821bb884f9cea07` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:32.622497 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d3d5be-3365-430f-9636-670f2a4c929e None admin] RBAC: Authorizing `identity:create_grant(project_id=f666a617f8d14eb18bbeb076032b82f2, user_id=ce508bb9aa164a7fb1831a9b5ff5b932, role_id=1c413f4e78ef45b7a821bb884f9cea07)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:32.624039 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d3d5be-3365-430f-9636-670f2a4c929e None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:32.637756 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 620/1232] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 12 14:03:32 2022] PUT /identity/v3/projects/f666a617f8d14eb18bbeb076032b82f2/users/ce508bb9aa164a7fb1831a9b5ff5b932/roles/1c413f4e78ef45b7a821bb884f9cea07 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:03:34.947046 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684db504-9a31-440c-b119-72126022b647 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:34.947612 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684db504-9a31-440c-b119-72126022b647 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:34.947612 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684db504-9a31-440c-b119-72126022b647 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:34.948921 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 613/1233] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:03:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:03:34.954094 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc0beb-7336-4e81-be6a-f8c327f55f7a None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:34.954545 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc0beb-7336-4e81-be6a-f8c327f55f7a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:34.954646 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc0beb-7336-4e81-be6a-f8c327f55f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:34.974890 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-5dbc0beb-7336-4e81-be6a-f8c327f55f7a None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:03:34.976168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-5dbc0beb-7336-4e81-be6a-f8c327f55f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:03:35.066111 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 621/1234] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:03:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:03:35.350098 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684db504-9a31-440c-b119-72126022b647 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:35.378624 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:35.464415 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684db504-9a31-440c-b119-72126022b647 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:35.514058 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a6b96-1d9e-4021-ad90-22baa6734e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:35.515192 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a6b96-1d9e-4021-ad90-22baa6734e16 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:35.515611 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a6b96-1d9e-4021-ad90-22baa6734e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:35.515931 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a6b96-1d9e-4021-ad90-22baa6734e16 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:35.530323 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844a6b96-1d9e-4021-ad90-22baa6734e16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:03:35.538419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-844a6b96-1d9e-4021-ad90-22baa6734e16 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:03:35.538419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:03:35.538419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:03:35.538419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:03:35.538419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:03:35.538419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:03:35.538419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:03:35.538419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:03:35.538419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:03:35.538419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:03:35.538419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:03:35.538419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:03:35.538419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:03:35.538419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:03:35.538419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:03:35.538419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:03:35.543254 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:03:35.543254 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:03:35.543254 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:03:35.543254 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:03:35.543254 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:03:35.543254 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:03:35.543254 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:03:35.543254 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:03:35.543254 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:03:35.543254 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:03:35.543254 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:03:35.543254 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:03:35.543254 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:03:35.543254 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:03:35.543254 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:03:35.543254 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:03:35.544395 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:03:35.544395 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:03:35.544395 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:03:35.544395 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:03:35.544395 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:03:35.544395 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:03:35.544395 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:03:35.544395 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:03:35.544395 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:03:35.544395 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:03:35.544395 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 14:03:35.544395 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:03:35.544395 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 614/1235] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 12 14:03:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 14:03:35.545711 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc0beb-7336-4e81-be6a-f8c327f55f7a None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:35.602495 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:35.662437 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc0beb-7336-4e81-be6a-f8c327f55f7a None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:35.700455 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3700c-e239-4c19-81ed-eb18fea12dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:35.701060 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3700c-e239-4c19-81ed-eb18fea12dfe None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:35.701336 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3700c-e239-4c19-81ed-eb18fea12dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:35.701603 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3700c-e239-4c19-81ed-eb18fea12dfe None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:35.702261 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb3700c-e239-4c19-81ed-eb18fea12dfe None admin] RBAC: Adding query filter params (name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:35.702558 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb3700c-e239-4c19-81ed-eb18fea12dfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:35.703936 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb3700c-e239-4c19-81ed-eb18fea12dfe None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:35.712305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 622/1236] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 12 14:03:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:03:35.718133 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a6b96-1d9e-4021-ad90-22baa6734e16 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:35.809774 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:35.833024 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a6b96-1d9e-4021-ad90-22baa6734e16 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:35.872920 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819de24e-981d-487a-b124-aef6c4012ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:35.873485 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819de24e-981d-487a-b124-aef6c4012ddc None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:35.873738 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819de24e-981d-487a-b124-aef6c4012ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:35.874011 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819de24e-981d-487a-b124-aef6c4012ddc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:35.884954 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819de24e-981d-487a-b124-aef6c4012ddc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:35.886542 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819de24e-981d-487a-b124-aef6c4012ddc None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:35.902193 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-819de24e-981d-487a-b124-aef6c4012ddc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:03:35.902193 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:03:35.902193 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:03:35.902193 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:03:35.902193 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:03:35.902193 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:03:35.902193 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:03:35.902193 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:03:35.902193 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:03:35.902193 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:03:35.902193 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:03:35.902193 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:03:35.902193 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:03:35.902193 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:03:35.902193 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:03:35.902193 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:03:35.903452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:03:35.903452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:03:35.903452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:03:35.903452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:03:35.903452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:03:35.903452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:03:35.903452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:03:35.903452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:03:35.903452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:03:35.903452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:03:35.903452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:03:35.903452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:03:35.903452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:03:35.903452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:03:35.903452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:03:35.903452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:03:35.903452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:03:35.904375 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:03:35.904375 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:03:35.904375 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:03:35.904375 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 615/1237] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:03:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:03:35.909204 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3700c-e239-4c19-81ed-eb18fea12dfe None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:36.015746 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3700c-e239-4c19-81ed-eb18fea12dfe None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:36.026874 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:36.054910 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd9a77-4c79-4df8-a385-2f589fb58670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:36.055552 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd9a77-4c79-4df8-a385-2f589fb58670 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:36.055959 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd9a77-4c79-4df8-a385-2f589fb58670 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:36.056269 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd9a77-4c79-4df8-a385-2f589fb58670 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:36.056975 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfd9a77-4c79-4df8-a385-2f589fb58670 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:36.057333 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfd9a77-4c79-4df8-a385-2f589fb58670 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:36.058807 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfd9a77-4c79-4df8-a385-2f589fb58670 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:36.072340 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 623/1238] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:03:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:03:36.078067 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819de24e-981d-487a-b124-aef6c4012ddc None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:36.181878 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819de24e-981d-487a-b124-aef6c4012ddc None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:36.220653 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792988a-b944-4f8b-a147-b0d89cb870ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:36.221385 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792988a-b944-4f8b-a147-b0d89cb870ae None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:36.221660 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792988a-b944-4f8b-a147-b0d89cb870ae None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:36.221926 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792988a-b944-4f8b-a147-b0d89cb870ae None admin] PATH_INFO: `/v3/users/placement` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:36.224633 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:36.232003 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8792988a-b944-4f8b-a147-b0d89cb870ae None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:36.233697 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8792988a-b944-4f8b-a147-b0d89cb870ae None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:36.246626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-8792988a-b944-4f8b-a147-b0d89cb870ae None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 12 14:03:36.246626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:03:36.246626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:03:36.246626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:03:36.246626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:03:36.246626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:03:36.246626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:03:36.246626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:03:36.246626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:03:36.246626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:03:36.246626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:03:36.246626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:03:36.246626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:03:36.246626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:03:36.246626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:03:36.246626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:03:36.247846 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:03:36.247846 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:03:36.247846 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:03:36.247846 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:03:36.247846 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:03:36.247846 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:03:36.247846 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:03:36.247846 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:03:36.247846 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:03:36.247846 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:03:36.247846 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:03:36.247846 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:03:36.247846 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:03:36.247846 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:03:36.247846 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:03:36.247846 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:03:36.247846 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:03:36.247846 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:03:36.248811 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:03:36.248811 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:03:36.248811 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:03:36.248811 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:03:36.248811 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:03:36.248811 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:03:36.248811 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:03:36.248811 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:03:36.248811 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 12 14:03:36.248811 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:03:36.248811 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 616/1239] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:03:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:03:36.254890 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd9a77-4c79-4df8-a385-2f589fb58670 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:36.367636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd9a77-4c79-4df8-a385-2f589fb58670 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:36.407851 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886316a-b65c-4223-a0de-45763872038b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:36.408462 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886316a-b65c-4223-a0de-45763872038b None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:36.408739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886316a-b65c-4223-a0de-45763872038b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:36.409008 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886316a-b65c-4223-a0de-45763872038b None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:36.409735 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e886316a-b65c-4223-a0de-45763872038b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:36.410038 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e886316a-b65c-4223-a0de-45763872038b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:36.411495 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e886316a-b65c-4223-a0de-45763872038b None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:36.429934 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 624/1240] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Sep 12 14:03:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:03:36.436859 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792988a-b944-4f8b-a147-b0d89cb870ae None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:36.548291 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792988a-b944-4f8b-a147-b0d89cb870ae None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:36.585037 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c71e9-77a9-464e-9b2e-65468b4af977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:36.585716 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c71e9-77a9-464e-9b2e-65468b4af977 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:36.586000 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c71e9-77a9-464e-9b2e-65468b4af977 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:36.586272 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c71e9-77a9-464e-9b2e-65468b4af977 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:36.657882 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:36.665569 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c71e9-77a9-464e-9b2e-65468b4af977 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:36.667090 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c71e9-77a9-464e-9b2e-65468b4af977 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:36.777114 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 617/1241] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:03:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 14:03:36.778720 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886316a-b65c-4223-a0de-45763872038b None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:36.808963 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:36.889083 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886316a-b65c-4223-a0de-45763872038b None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:36.931501 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babcf2ed-f0f9-49f4-a6a1-7d5f4e84b7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:36.932131 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babcf2ed-f0f9-49f4-a6a1-7d5f4e84b7f6 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:36.932413 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babcf2ed-f0f9-49f4-a6a1-7d5f4e84b7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:36.932689 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babcf2ed-f0f9-49f4-a6a1-7d5f4e84b7f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:36.943674 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babcf2ed-f0f9-49f4-a6a1-7d5f4e84b7f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:36.945107 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babcf2ed-f0f9-49f4-a6a1-7d5f4e84b7f6 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:36.961739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-babcf2ed-f0f9-49f4-a6a1-7d5f4e84b7f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:03:36.961739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:03:36.961739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:03:36.961739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:03:36.961739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:03:36.961739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:03:36.961739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:03:36.961739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:03:36.961739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:03:36.961739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:03:36.961739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:03:36.961739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:03:36.961739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:03:36.961739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:03:36.961739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:03:36.961739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:03:36.963148 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:03:36.963148 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:03:36.963148 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:03:36.963148 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:03:36.963148 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:03:36.963148 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:03:36.963148 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:03:36.963148 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:03:36.963148 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:03:36.963148 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:03:36.963148 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:03:36.963148 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:03:36.963148 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:03:36.963148 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:03:36.963148 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:03:36.963148 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:03:36.963148 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:03:36.964861 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:03:36.964861 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:03:36.964861 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:03:36.964861 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 625/1242] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:03:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:03:36.968809 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c71e9-77a9-464e-9b2e-65468b4af977 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:37.069121 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:37.075673 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c71e9-77a9-464e-9b2e-65468b4af977 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:37.115438 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b6a04-b346-485d-9363-c55d6cac1157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:37.116131 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b6a04-b346-485d-9363-c55d6cac1157 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:37.116319 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b6a04-b346-485d-9363-c55d6cac1157 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:37.116578 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b6a04-b346-485d-9363-c55d6cac1157 None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:37.117217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157b6a04-b346-485d-9363-c55d6cac1157 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:37.117514 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157b6a04-b346-485d-9363-c55d6cac1157 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:37.118891 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157b6a04-b346-485d-9363-c55d6cac1157 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:37.132930 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 618/1243] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:03:37.139025 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babcf2ed-f0f9-49f4-a6a1-7d5f4e84b7f6 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:37.225112 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:37.252799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babcf2ed-f0f9-49f4-a6a1-7d5f4e84b7f6 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:37.288850 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad936ec-0024-46c0-9b65-f2fe22a73a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:37.289709 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad936ec-0024-46c0-9b65-f2fe22a73a41 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:37.289998 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad936ec-0024-46c0-9b65-f2fe22a73a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:37.290253 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad936ec-0024-46c0-9b65-f2fe22a73a41 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:37.298566 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad936ec-0024-46c0-9b65-f2fe22a73a41 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:37.300853 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad936ec-0024-46c0-9b65-f2fe22a73a41 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:37.314127 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-2ad936ec-0024-46c0-9b65-f2fe22a73a41 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:03:37.314127 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:03:37.314127 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:03:37.314127 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:03:37.314127 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:03:37.314127 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:03:37.314127 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:03:37.314127 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:03:37.314127 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:03:37.314127 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:03:37.314127 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:03:37.314127 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:03:37.314127 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:03:37.314127 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:03:37.314127 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:03:37.316327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:03:37.316327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:03:37.316327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:03:37.316327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:03:37.316327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:03:37.316327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:03:37.316327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:03:37.316327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:03:37.316327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:03:37.316327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:03:37.316327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:03:37.316327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:03:37.316327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:03:37.316327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:03:37.316327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:03:37.316327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:03:37.316327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:03:37.317517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:03:37.317517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:03:37.317517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:03:37.317517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:03:37.317517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:03:37.317517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:03:37.317517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:03:37.317517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:03:37.317517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 626/1244] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:03:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:03:37.322138 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b6a04-b346-485d-9363-c55d6cac1157 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:37.438261 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b6a04-b346-485d-9363-c55d6cac1157 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:37.479898 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab71d2bf-9de7-4107-b63c-4c0d259e5437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:37.480728 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab71d2bf-9de7-4107-b63c-4c0d259e5437 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:37.481161 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab71d2bf-9de7-4107-b63c-4c0d259e5437 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:37.481486 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab71d2bf-9de7-4107-b63c-4c0d259e5437 None admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:37.482643 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab71d2bf-9de7-4107-b63c-4c0d259e5437 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:37.483210 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab71d2bf-9de7-4107-b63c-4c0d259e5437 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:37.485241 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab71d2bf-9de7-4107-b63c-4c0d259e5437 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:37.505071 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 619/1245] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 12 14:03:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:03:37.513513 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad936ec-0024-46c0-9b65-f2fe22a73a41 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:37.638468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad936ec-0024-46c0-9b65-f2fe22a73a41 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:37.680717 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eb3a7b-0e6f-4837-9202-72e4098ea290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:37.681663 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb3a7b-0e6f-4837-9202-72e4098ea290 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:37.682067 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb3a7b-0e6f-4837-9202-72e4098ea290 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:37.682311 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb3a7b-0e6f-4837-9202-72e4098ea290 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:37.683291 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eb3a7b-0e6f-4837-9202-72e4098ea290 None admin] RBAC: Adding query filter params (role.id=1c413f4e78ef45b7a821bb884f9cea07, scope.project.id=f666a617f8d14eb18bbeb076032b82f2, user.id=ce508bb9aa164a7fb1831a9b5ff5b932) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:37.683691 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eb3a7b-0e6f-4837-9202-72e4098ea290 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:37.685354 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eb3a7b-0e6f-4837-9202-72e4098ea290 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:37.692459 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 627/1246] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 14:03:37 2022] GET /identity/v3/role_assignments?user.id=ce508bb9aa164a7fb1831a9b5ff5b932&scope.project.id=f666a617f8d14eb18bbeb076032b82f2&role.id=1c413f4e78ef45b7a821bb884f9cea07 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:03:39.987753 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944ea76-345b-41c9-a630-b75bc2db36a5 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:39.988292 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944ea76-345b-41c9-a630-b75bc2db36a5 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:39.988292 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944ea76-345b-41c9-a630-b75bc2db36a5 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:39.989630 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 620/1247] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:03:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:03:39.998518 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d110821-43c8-4a2f-b3f5-547726714d67 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:39.998518 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d110821-43c8-4a2f-b3f5-547726714d67 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:39.998518 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d110821-43c8-4a2f-b3f5-547726714d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:40.002561 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:40.015523 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-2d110821-43c8-4a2f-b3f5-547726714d67 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:03:40.016730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-2d110821-43c8-4a2f-b3f5-547726714d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:03:40.089051 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 628/1248] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:03:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 14:03:40.382691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4944ea76-345b-41c9-a630-b75bc2db36a5 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:40.401349 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:40.496308 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4944ea76-345b-41c9-a630-b75bc2db36a5 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:40.539220 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806563c3-97a8-418c-a544-fba096aac37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:40.539847 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806563c3-97a8-418c-a544-fba096aac37e None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:40.540129 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806563c3-97a8-418c-a544-fba096aac37e None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:40.540421 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806563c3-97a8-418c-a544-fba096aac37e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:40.553050 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806563c3-97a8-418c-a544-fba096aac37e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:03:40.558275 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-806563c3-97a8-418c-a544-fba096aac37e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:03:40.558275 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:03:40.558275 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:03:40.558275 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:03:40.558275 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:03:40.558275 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:03:40.558275 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:03:40.558275 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:03:40.558275 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:03:40.558275 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:03:40.558275 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:03:40.558275 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:03:40.558275 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:03:40.558275 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:03:40.558275 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:03:40.558275 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:03:40.559556 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:03:40.559556 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:03:40.559556 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:03:40.559556 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:03:40.559556 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:03:40.559556 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:03:40.559556 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:03:40.559556 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:03:40.559556 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:03:40.559556 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:03:40.559556 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:03:40.559556 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:03:40.559556 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:03:40.559556 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:03:40.559556 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:03:40.559556 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:03:40.561162 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:03:40.561162 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:03:40.561162 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:03:40.561162 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:03:40.561162 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:03:40.561162 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:03:40.561162 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:03:40.561162 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:03:40.561162 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:03:40.561162 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:03:40.561162 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:03:40.561162 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:03:40.561162 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 621/1249] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 12 14:03:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:03:40.567877 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d110821-43c8-4a2f-b3f5-547726714d67 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:40.671427 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d110821-43c8-4a2f-b3f5-547726714d67 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:40.702868 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:40.709573 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51dd0d-0d2d-464a-bf79-8752238974fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:40.710253 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51dd0d-0d2d-464a-bf79-8752238974fd None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:40.710651 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51dd0d-0d2d-464a-bf79-8752238974fd None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:40.710953 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51dd0d-0d2d-464a-bf79-8752238974fd None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:40.711761 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a51dd0d-0d2d-464a-bf79-8752238974fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:40.712104 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a51dd0d-0d2d-464a-bf79-8752238974fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:40.713644 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a51dd0d-0d2d-464a-bf79-8752238974fd None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:40.721671 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 629/1250] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 14:03:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:03:40.727498 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806563c3-97a8-418c-a544-fba096aac37e None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:40.817080 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:40.848116 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806563c3-97a8-418c-a544-fba096aac37e None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:40.887318 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee1229-630c-4f57-b4d6-200efff1a25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:40.887970 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee1229-630c-4f57-b4d6-200efff1a25d None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:40.888245 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee1229-630c-4f57-b4d6-200efff1a25d None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:40.888513 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee1229-630c-4f57-b4d6-200efff1a25d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:40.900864 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee1229-630c-4f57-b4d6-200efff1a25d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:40.902336 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee1229-630c-4f57-b4d6-200efff1a25d None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:40.917780 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-a6ee1229-630c-4f57-b4d6-200efff1a25d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:03:40.917780 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:03:40.917780 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:03:40.917780 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:03:40.917780 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:03:40.917780 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:03:40.917780 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:03:40.917780 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:03:40.917780 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:03:40.917780 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:03:40.917780 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:03:40.917780 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:03:40.917780 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:03:40.917780 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:03:40.917780 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:03:40.917780 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:03:40.919002 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:03:40.919002 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:03:40.919002 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:03:40.919002 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:03:40.919002 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:03:40.919002 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:03:40.919002 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:03:40.919002 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:03:40.919002 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:03:40.919002 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:03:40.919002 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:03:40.919002 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:03:40.919002 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:03:40.919002 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:03:40.919002 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:03:40.919002 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:03:40.919002 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:03:40.919917 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:03:40.919917 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:03:40.919917 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:03:40.920486 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 622/1251] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:03:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:03:40.925510 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51dd0d-0d2d-464a-bf79-8752238974fd None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:41.029691 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51dd0d-0d2d-464a-bf79-8752238974fd None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:41.065392 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e660f9b-a7d4-4f10-8d8a-cf8402501f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:41.066005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e660f9b-a7d4-4f10-8d8a-cf8402501f79 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:41.066285 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e660f9b-a7d4-4f10-8d8a-cf8402501f79 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:41.066558 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e660f9b-a7d4-4f10-8d8a-cf8402501f79 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:41.067249 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e660f9b-a7d4-4f10-8d8a-cf8402501f79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:41.067553 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e660f9b-a7d4-4f10-8d8a-cf8402501f79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:41.068947 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e660f9b-a7d4-4f10-8d8a-cf8402501f79 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:41.087017 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 630/1252] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:03:41.088248 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee1229-630c-4f57-b4d6-200efff1a25d None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:41.199972 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee1229-630c-4f57-b4d6-200efff1a25d None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:41.226150 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:41.234527 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a154e4c-ef30-4146-9462-e44e72fba5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:41.235193 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a154e4c-ef30-4146-9462-e44e72fba5e1 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:41.235467 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a154e4c-ef30-4146-9462-e44e72fba5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:41.235733 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a154e4c-ef30-4146-9462-e44e72fba5e1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:41.244363 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a154e4c-ef30-4146-9462-e44e72fba5e1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:41.245812 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a154e4c-ef30-4146-9462-e44e72fba5e1 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:41.258308 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-6a154e4c-ef30-4146-9462-e44e72fba5e1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 12 14:03:41.258308 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:03:41.258308 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:03:41.258308 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:03:41.258308 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:03:41.258308 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:03:41.258308 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:03:41.258308 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:03:41.258308 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:03:41.258308 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:03:41.258308 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:03:41.258308 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:03:41.258308 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:03:41.258308 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:03:41.258308 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:03:41.258308 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:03:41.259615 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:03:41.259615 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:03:41.259615 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:03:41.259615 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:03:41.259615 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:03:41.259615 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:03:41.259615 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:03:41.259615 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:03:41.259615 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:03:41.259615 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:03:41.259615 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:03:41.259615 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:03:41.259615 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:03:41.259615 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:03:41.259615 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:03:41.259615 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:03:41.259615 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:03:41.259615 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:03:41.261038 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:03:41.261038 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:03:41.261038 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:03:41.261038 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:03:41.261038 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:03:41.261038 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:03:41.261038 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:03:41.261038 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:03:41.261038 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 12 14:03:41.261038 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:03:41.261038 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 623/1253] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:03:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:03:41.266975 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e660f9b-a7d4-4f10-8d8a-cf8402501f79 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:41.323504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:41.368864 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e660f9b-a7d4-4f10-8d8a-cf8402501f79 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:41.402149 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de50d9e-47fa-4474-b719-53b704668649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:41.402762 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de50d9e-47fa-4474-b719-53b704668649 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:41.403074 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de50d9e-47fa-4474-b719-53b704668649 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:41.403350 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de50d9e-47fa-4474-b719-53b704668649 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:41.404078 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de50d9e-47fa-4474-b719-53b704668649 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:41.404385 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de50d9e-47fa-4474-b719-53b704668649 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:41.405826 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de50d9e-47fa-4474-b719-53b704668649 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:41.423644 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 631/1254] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Sep 12 14:03:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:03:41.429704 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a154e4c-ef30-4146-9462-e44e72fba5e1 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:41.547605 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a154e4c-ef30-4146-9462-e44e72fba5e1 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:41.581395 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450ea16-c99f-430a-8d88-af9bba0ca868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:41.581989 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450ea16-c99f-430a-8d88-af9bba0ca868 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:41.582266 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450ea16-c99f-430a-8d88-af9bba0ca868 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:41.582535 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450ea16-c99f-430a-8d88-af9bba0ca868 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:41.635310 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:41.648031 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8450ea16-c99f-430a-8d88-af9bba0ca868 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:41.649465 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8450ea16-c99f-430a-8d88-af9bba0ca868 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:41.743173 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 624/1255] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:03:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 14:03:41.748981 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de50d9e-47fa-4474-b719-53b704668649 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:41.848182 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de50d9e-47fa-4474-b719-53b704668649 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:41.860545 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:41.880546 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb2787-57fa-4817-a2c8-343d24239288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:41.881161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb2787-57fa-4817-a2c8-343d24239288 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:41.881434 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb2787-57fa-4817-a2c8-343d24239288 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:41.881699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb2787-57fa-4817-a2c8-343d24239288 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:41.891117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb2787-57fa-4817-a2c8-343d24239288 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:41.893070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb2787-57fa-4817-a2c8-343d24239288 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:41.906890 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-9afb2787-57fa-4817-a2c8-343d24239288 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:03:41.906890 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:03:41.906890 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:03:41.906890 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:03:41.906890 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:03:41.906890 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:03:41.906890 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:03:41.906890 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:03:41.906890 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:03:41.906890 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:03:41.906890 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:03:41.906890 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:03:41.906890 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:03:41.906890 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:03:41.906890 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:03:41.906890 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:03:41.907987 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:03:41.907987 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:03:41.907987 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:03:41.907987 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:03:41.907987 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:03:41.907987 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:03:41.907987 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:03:41.907987 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:03:41.907987 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:03:41.907987 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:03:41.907987 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:03:41.907987 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:03:41.907987 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:03:41.907987 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:03:41.907987 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:03:41.907987 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:03:41.907987 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:03:41.909776 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:03:41.909776 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:03:41.909776 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:03:41.909776 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 632/1256] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:03:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:03:41.914216 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450ea16-c99f-430a-8d88-af9bba0ca868 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:41.996515 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:42.009267 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450ea16-c99f-430a-8d88-af9bba0ca868 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:42.043069 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee849ad-e153-43ee-bd25-7eca3607ac5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:42.043656 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee849ad-e153-43ee-bd25-7eca3607ac5b None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:42.043927 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee849ad-e153-43ee-bd25-7eca3607ac5b None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:42.044188 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee849ad-e153-43ee-bd25-7eca3607ac5b None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:42.044808 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee849ad-e153-43ee-bd25-7eca3607ac5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:42.045100 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee849ad-e153-43ee-bd25-7eca3607ac5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:42.046430 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee849ad-e153-43ee-bd25-7eca3607ac5b None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:42.056981 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 625/1257] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:03:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:03:42.062699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb2787-57fa-4817-a2c8-343d24239288 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:42.159299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb2787-57fa-4817-a2c8-343d24239288 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:42.190800 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92956db-f739-44ea-bbfd-9122582234d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:42.191435 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92956db-f739-44ea-bbfd-9122582234d8 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:42.191709 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92956db-f739-44ea-bbfd-9122582234d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:42.191975 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92956db-f739-44ea-bbfd-9122582234d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:42.198515 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92956db-f739-44ea-bbfd-9122582234d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:42.199955 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92956db-f739-44ea-bbfd-9122582234d8 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:42.211373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-c92956db-f739-44ea-bbfd-9122582234d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:03:42.211373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:03:42.211373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:03:42.211373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:03:42.211373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:03:42.211373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:03:42.211373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:03:42.211373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:03:42.211373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:03:42.211373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:03:42.211373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:03:42.211373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:03:42.211373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:03:42.211373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:03:42.211373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:03:42.212440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:03:42.212440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:03:42.212440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:03:42.212440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:03:42.212440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:03:42.212440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:03:42.212440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:03:42.212440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:03:42.212440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:03:42.212440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:03:42.212440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:03:42.212440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:03:42.212440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:03:42.212440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:03:42.212440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:03:42.212440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:03:42.212440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:03:42.213370 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:03:42.213370 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:03:42.213370 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:03:42.213370 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:03:42.213370 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:03:42.213370 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:03:42.213370 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:03:42.213370 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:03:42.213370 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 633/1258] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:03:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:03:42.218272 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee849ad-e153-43ee-bd25-7eca3607ac5b None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:42.312419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee849ad-e153-43ee-bd25-7eca3607ac5b None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:42.345041 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca54a6c-5bf9-42c1-9ec8-ba18535cb75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:42.345665 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca54a6c-5bf9-42c1-9ec8-ba18535cb75e None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:42.345936 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca54a6c-5bf9-42c1-9ec8-ba18535cb75e None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:42.346200 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca54a6c-5bf9-42c1-9ec8-ba18535cb75e None admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:42.346881 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca54a6c-5bf9-42c1-9ec8-ba18535cb75e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:42.347178 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca54a6c-5bf9-42c1-9ec8-ba18535cb75e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:42.348560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca54a6c-5bf9-42c1-9ec8-ba18535cb75e None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:42.359847 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 626/1259] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 12 14:03:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:03:42.365553 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92956db-f739-44ea-bbfd-9122582234d8 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:42.390803 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:42.465891 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92956db-f739-44ea-bbfd-9122582234d8 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:42.500020 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd24317-9037-4103-8627-ed1d9c2ae778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:42.500648 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd24317-9037-4103-8627-ed1d9c2ae778 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:42.500909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd24317-9037-4103-8627-ed1d9c2ae778 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:42.501180 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd24317-9037-4103-8627-ed1d9c2ae778 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:42.502445 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd24317-9037-4103-8627-ed1d9c2ae778 None admin] RBAC: Adding query filter params (role.id=e14564f00daf48a28d4e32da8cd2efad, scope.project.id=f666a617f8d14eb18bbeb076032b82f2, user.id=ce508bb9aa164a7fb1831a9b5ff5b932) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:42.502818 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd24317-9037-4103-8627-ed1d9c2ae778 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:42.504204 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd24317-9037-4103-8627-ed1d9c2ae778 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:42.509726 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 634/1260] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 14:03:42 2022] GET /identity/v3/role_assignments?user.id=ce508bb9aa164a7fb1831a9b5ff5b932&scope.project.id=f666a617f8d14eb18bbeb076032b82f2&role.id=e14564f00daf48a28d4e32da8cd2efad => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:03:44.611998 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84125575-d92a-4df6-90bb-28ee897b7f93 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:44.612525 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84125575-d92a-4df6-90bb-28ee897b7f93 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:44.612525 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84125575-d92a-4df6-90bb-28ee897b7f93 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:44.613862 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 627/1261] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:03:44.619009 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec9388-1919-43f0-a3db-d774af971648 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:44.619410 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec9388-1919-43f0-a3db-d774af971648 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:44.619566 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec9388-1919-43f0-a3db-d774af971648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:44.638511 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-63ec9388-1919-43f0-a3db-d774af971648 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:03:44.639726 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-63ec9388-1919-43f0-a3db-d774af971648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:03:44.705844 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 635/1262] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:03:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:03:44.972058 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84125575-d92a-4df6-90bb-28ee897b7f93 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:44.982686 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:45.069583 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84125575-d92a-4df6-90bb-28ee897b7f93 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:45.102561 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bab0fa8-7f72-45ce-807f-e79997b7d194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:45.103267 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab0fa8-7f72-45ce-807f-e79997b7d194 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:45.103541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab0fa8-7f72-45ce-807f-e79997b7d194 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:45.103803 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab0fa8-7f72-45ce-807f-e79997b7d194 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:45.113900 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bab0fa8-7f72-45ce-807f-e79997b7d194 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:03:45.118651 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-4bab0fa8-7f72-45ce-807f-e79997b7d194 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:03:45.118651 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:03:45.118651 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:03:45.118651 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:03:45.118651 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:03:45.118651 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:03:45.118651 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:03:45.118651 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:03:45.118651 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:03:45.118651 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:03:45.118651 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:03:45.118651 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:03:45.118651 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:03:45.118651 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:03:45.118651 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:03:45.118651 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:03:45.119706 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:03:45.119706 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:03:45.119706 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:03:45.119706 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:03:45.119706 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:03:45.119706 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:03:45.119706 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:03:45.119706 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:03:45.119706 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:03:45.119706 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:03:45.119706 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:03:45.119706 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:03:45.119706 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:03:45.119706 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:03:45.119706 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:03:45.119706 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:03:45.120633 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:03:45.120633 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:03:45.120633 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:03:45.120633 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:03:45.120633 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:03:45.120633 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:03:45.120633 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:03:45.120633 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:03:45.120633 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:03:45.120633 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:03:45.120633 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:03:45.120633 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:03:45.120633 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 628/1263] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 12 14:03:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:03:45.125876 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec9388-1919-43f0-a3db-d774af971648 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:45.143346 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:45.221832 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec9388-1919-43f0-a3db-d774af971648 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:45.254104 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd697bc3-b4cf-4f2a-b985-6dae077f42e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:45.254646 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd697bc3-b4cf-4f2a-b985-6dae077f42e5 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:45.254947 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd697bc3-b4cf-4f2a-b985-6dae077f42e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:45.255217 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd697bc3-b4cf-4f2a-b985-6dae077f42e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:45.255869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd697bc3-b4cf-4f2a-b985-6dae077f42e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:45.256165 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd697bc3-b4cf-4f2a-b985-6dae077f42e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:45.257490 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd697bc3-b4cf-4f2a-b985-6dae077f42e5 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:45.265112 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 636/1264] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 14:03:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:03:45.270867 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bab0fa8-7f72-45ce-807f-e79997b7d194 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:45.335578 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:45.368455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bab0fa8-7f72-45ce-807f-e79997b7d194 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:45.401139 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade48660-6769-47d1-b949-618e0b02ec99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:45.401759 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade48660-6769-47d1-b949-618e0b02ec99 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:45.402035 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade48660-6769-47d1-b949-618e0b02ec99 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:45.402305 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade48660-6769-47d1-b949-618e0b02ec99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:45.412646 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade48660-6769-47d1-b949-618e0b02ec99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:45.414082 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade48660-6769-47d1-b949-618e0b02ec99 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:45.428250 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-ade48660-6769-47d1-b949-618e0b02ec99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:03:45.428250 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:03:45.428250 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:03:45.428250 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:03:45.428250 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:03:45.428250 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:03:45.428250 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:03:45.428250 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:03:45.428250 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:03:45.428250 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:03:45.428250 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:03:45.428250 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:03:45.428250 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:03:45.428250 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:03:45.428250 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:03:45.428250 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:03:45.429167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:03:45.429167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:03:45.429167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:03:45.429167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:03:45.429167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:03:45.429167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:03:45.429167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:03:45.429167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:03:45.429167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:03:45.429167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:03:45.429167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:03:45.429167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:03:45.429167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:03:45.429167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:03:45.429167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:03:45.429167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:03:45.429167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:03:45.430085 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:03:45.430085 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:03:45.430085 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:03:45.430085 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 629/1265] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:03:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:03:45.434969 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd697bc3-b4cf-4f2a-b985-6dae077f42e5 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:45.518423 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:45.539138 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd697bc3-b4cf-4f2a-b985-6dae077f42e5 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:45.575877 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64f29ea-ef80-417c-a355-392dda00bef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:45.576472 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f29ea-ef80-417c-a355-392dda00bef8 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:45.576749 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f29ea-ef80-417c-a355-392dda00bef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:45.577019 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f29ea-ef80-417c-a355-392dda00bef8 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:45.577652 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64f29ea-ef80-417c-a355-392dda00bef8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:45.577949 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64f29ea-ef80-417c-a355-392dda00bef8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:45.579335 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64f29ea-ef80-417c-a355-392dda00bef8 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:45.591163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 637/1266] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:03:45.597609 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade48660-6769-47d1-b949-618e0b02ec99 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:45.694304 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade48660-6769-47d1-b949-618e0b02ec99 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:45.713397 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:45.727579 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e1151-7420-47dd-82bf-7afc84134363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:45.728187 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e1151-7420-47dd-82bf-7afc84134363 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:45.728459 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e1151-7420-47dd-82bf-7afc84134363 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:45.728725 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e1151-7420-47dd-82bf-7afc84134363 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:45.737035 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13e1151-7420-47dd-82bf-7afc84134363 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:45.738450 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13e1151-7420-47dd-82bf-7afc84134363 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:45.749176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-d13e1151-7420-47dd-82bf-7afc84134363 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 12 14:03:45.749176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:03:45.749176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:03:45.749176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:03:45.749176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:03:45.749176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:03:45.749176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:03:45.749176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:03:45.749176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:03:45.749176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:03:45.749176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:03:45.749176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:03:45.749176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:03:45.749176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:03:45.749176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:03:45.749176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:03:45.750258 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:03:45.750258 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:03:45.750258 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:03:45.750258 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:03:45.750258 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:03:45.750258 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:03:45.750258 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:03:45.750258 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:03:45.750258 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:03:45.750258 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:03:45.750258 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:03:45.750258 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:03:45.750258 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:03:45.750258 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:03:45.750258 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:03:45.750258 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:03:45.750258 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:03:45.750258 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:03:45.751470 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:03:45.751470 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:03:45.751470 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:03:45.751470 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:03:45.751470 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:03:45.751470 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:03:45.751470 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:03:45.751470 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:03:45.751470 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 12 14:03:45.751470 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:03:45.751470 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 630/1267] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:03:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:03:45.756179 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64f29ea-ef80-417c-a355-392dda00bef8 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:45.852300 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64f29ea-ef80-417c-a355-392dda00bef8 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:45.884523 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e87a8-793c-44d4-9b99-d0fc8f97d2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:45.885118 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e87a8-793c-44d4-9b99-d0fc8f97d2ad None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:45.885395 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e87a8-793c-44d4-9b99-d0fc8f97d2ad None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:45.885664 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e87a8-793c-44d4-9b99-d0fc8f97d2ad None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:45.886382 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e87a8-793c-44d4-9b99-d0fc8f97d2ad None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:45.886682 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e87a8-793c-44d4-9b99-d0fc8f97d2ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:45.888104 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e87a8-793c-44d4-9b99-d0fc8f97d2ad None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:45.896893 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:45.905276 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 638/1268] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Sep 12 14:03:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:03:45.911133 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e1151-7420-47dd-82bf-7afc84134363 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:46.006341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e1151-7420-47dd-82bf-7afc84134363 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:46.038020 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a436dca-c3cc-4316-a6d2-7c31f46d6eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:46.038616 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a436dca-c3cc-4316-a6d2-7c31f46d6eee None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:46.038917 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a436dca-c3cc-4316-a6d2-7c31f46d6eee None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:46.039182 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a436dca-c3cc-4316-a6d2-7c31f46d6eee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:46.049053 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a436dca-c3cc-4316-a6d2-7c31f46d6eee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:46.050460 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a436dca-c3cc-4316-a6d2-7c31f46d6eee None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:46.063871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-5a436dca-c3cc-4316-a6d2-7c31f46d6eee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:03:46.063871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:03:46.063871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:03:46.063871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:03:46.063871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:03:46.063871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:03:46.063871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:03:46.063871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:03:46.063871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:03:46.063871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:03:46.063871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:03:46.063871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:03:46.063871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:03:46.063871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:03:46.063871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:03:46.063871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:03:46.065438 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:03:46.065438 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:03:46.065438 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:03:46.065438 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:03:46.065438 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:03:46.065438 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:03:46.065438 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:03:46.065438 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:03:46.065438 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:03:46.065438 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:03:46.065438 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:03:46.065438 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:03:46.065438 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:03:46.065438 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:03:46.065438 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:03:46.065438 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:03:46.065438 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:03:46.066333 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:03:46.066333 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:03:46.066333 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:03:46.067119 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 631/1269] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:03:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:03:46.073230 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e87a8-793c-44d4-9b99-d0fc8f97d2ad None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:46.168436 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e87a8-793c-44d4-9b99-d0fc8f97d2ad None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:46.201703 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f37dd-ba3f-472e-b453-a7e239328554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:46.202293 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f37dd-ba3f-472e-b453-a7e239328554 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:46.202647 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f37dd-ba3f-472e-b453-a7e239328554 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:46.202955 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f37dd-ba3f-472e-b453-a7e239328554 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:46.203597 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f37dd-ba3f-472e-b453-a7e239328554 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:46.203896 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f37dd-ba3f-472e-b453-a7e239328554 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:46.205228 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f37dd-ba3f-472e-b453-a7e239328554 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:46.215984 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 639/1270] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:03:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:03:46.221482 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a436dca-c3cc-4316-a6d2-7c31f46d6eee None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:46.252364 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:46.317452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a436dca-c3cc-4316-a6d2-7c31f46d6eee None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:46.348913 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7657629-3ba9-4d5b-a7fe-5c4a2c550c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:46.349507 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7657629-3ba9-4d5b-a7fe-5c4a2c550c6a None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:46.349782 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7657629-3ba9-4d5b-a7fe-5c4a2c550c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:46.350047 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7657629-3ba9-4d5b-a7fe-5c4a2c550c6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:46.356280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7657629-3ba9-4d5b-a7fe-5c4a2c550c6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:46.357691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7657629-3ba9-4d5b-a7fe-5c4a2c550c6a None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:46.368898 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-b7657629-3ba9-4d5b-a7fe-5c4a2c550c6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:03:46.368898 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:03:46.368898 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:03:46.368898 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:03:46.368898 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:03:46.368898 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:03:46.368898 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:03:46.368898 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:03:46.368898 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:03:46.368898 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:03:46.368898 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:03:46.368898 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:03:46.368898 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:03:46.368898 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:03:46.368898 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:03:46.369949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:03:46.369949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:03:46.369949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:03:46.369949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:03:46.369949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:03:46.369949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:03:46.369949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:03:46.369949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:03:46.369949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:03:46.369949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:03:46.369949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:03:46.369949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:03:46.369949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:03:46.369949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:03:46.369949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:03:46.369949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:03:46.369949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:03:46.371197 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:03:46.371197 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:03:46.371197 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:03:46.371197 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:03:46.371197 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:03:46.371197 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:03:46.371197 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:03:46.371197 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:03:46.371197 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 632/1271] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:03:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:03:46.375857 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f37dd-ba3f-472e-b453-a7e239328554 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:46.433144 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:46.473327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f37dd-ba3f-472e-b453-a7e239328554 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:46.507336 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f8aab-3c08-4e9d-a0f0-415bb829f4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:46.507934 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f8aab-3c08-4e9d-a0f0-415bb829f4a2 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:46.508212 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f8aab-3c08-4e9d-a0f0-415bb829f4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:46.508481 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f8aab-3c08-4e9d-a0f0-415bb829f4a2 None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:46.509227 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3f8aab-3c08-4e9d-a0f0-415bb829f4a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:46.509469 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3f8aab-3c08-4e9d-a0f0-415bb829f4a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:46.510889 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3f8aab-3c08-4e9d-a0f0-415bb829f4a2 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:46.523027 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 640/1272] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 12 14:03:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:03:46.528951 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7657629-3ba9-4d5b-a7fe-5c4a2c550c6a None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:46.617601 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:46.626407 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7657629-3ba9-4d5b-a7fe-5c4a2c550c6a None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:46.658796 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc79987-2a70-412f-998a-900c2f180d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:46.659461 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc79987-2a70-412f-998a-900c2f180d53 None admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:46.659734 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc79987-2a70-412f-998a-900c2f180d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:46.660001 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc79987-2a70-412f-998a-900c2f180d53 None admin] PATH_INFO: `/v3/projects/f666a617f8d14eb18bbeb076032b82f2/users/ce508bb9aa164a7fb1831a9b5ff5b932/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:46.661841 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc79987-2a70-412f-998a-900c2f180d53 None admin] RBAC: Authorizing `identity:create_grant(project_id=f666a617f8d14eb18bbeb076032b82f2, user_id=ce508bb9aa164a7fb1831a9b5ff5b932, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:46.663286 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc79987-2a70-412f-998a-900c2f180d53 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:46.675771 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 633/1273] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 12 14:03:46 2022] PUT /identity/v3/projects/f666a617f8d14eb18bbeb076032b82f2/users/ce508bb9aa164a7fb1831a9b5ff5b932/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:03:48.793971 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694ac60-0635-4b0c-9227-ebaf9335501d None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:48.794554 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694ac60-0635-4b0c-9227-ebaf9335501d None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:48.794554 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694ac60-0635-4b0c-9227-ebaf9335501d None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:48.795881 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 641/1274] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:03:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:03:48.801679 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6ca11-01c9-4366-b78e-417c545dd703 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:48.802105 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6ca11-01c9-4366-b78e-417c545dd703 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:48.802218 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6ca11-01c9-4366-b78e-417c545dd703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:48.819984 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-07d6ca11-01c9-4366-b78e-417c545dd703 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:03:48.821435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-07d6ca11-01c9-4366-b78e-417c545dd703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:03:48.889845 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 634/1275] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:03:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:03:49.160300 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2694ac60-0635-4b0c-9227-ebaf9335501d None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:49.264332 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2694ac60-0635-4b0c-9227-ebaf9335501d None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:49.280483 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:49.296648 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f47e54-6735-4631-852d-602f96bb5d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:49.297262 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f47e54-6735-4631-852d-602f96bb5d5d None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:49.297541 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f47e54-6735-4631-852d-602f96bb5d5d None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:49.297811 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f47e54-6735-4631-852d-602f96bb5d5d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:49.307924 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f47e54-6735-4631-852d-602f96bb5d5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 14:03:49.312671 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-d1f47e54-6735-4631-852d-602f96bb5d5d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:03:49.312671 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:03:49.312671 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:03:49.312671 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:03:49.312671 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:03:49.312671 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:03:49.312671 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:03:49.312671 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:03:49.312671 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:03:49.312671 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:03:49.312671 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:03:49.312671 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:03:49.312671 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 14:03:49.312671 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 14:03:49.312671 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 14:03:49.312671 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 14:03:49.313775 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 14:03:49.313775 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 14:03:49.313775 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 14:03:49.313775 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 14:03:49.313775 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:03:49.313775 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:03:49.313775 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:03:49.313775 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:03:49.313775 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:03:49.313775 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:03:49.313775 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:03:49.313775 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:03:49.313775 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:03:49.313775 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:03:49.313775 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:03:49.313775 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:03:49.315239 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:03:49.315239 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:03:49.315239 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:03:49.315239 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:03:49.315239 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 14:03:49.315239 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 14:03:49.315239 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 14:03:49.315239 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 14:03:49.315239 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 14:03:49.315239 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 14:03:49.315239 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 14:03:49.315239 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:03:49.315239 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 642/1276] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 12 14:03:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:03:49.320670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d6ca11-01c9-4366-b78e-417c545dd703 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:49.397848 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:49.416440 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d6ca11-01c9-4366-b78e-417c545dd703 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:49.450243 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8f46b-a957-4dd9-a44b-c55c200b7257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:49.450832 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8f46b-a957-4dd9-a44b-c55c200b7257 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:49.451133 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8f46b-a957-4dd9-a44b-c55c200b7257 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:49.451399 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8f46b-a957-4dd9-a44b-c55c200b7257 None admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:49.452053 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8f46b-a957-4dd9-a44b-c55c200b7257 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:49.452347 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8f46b-a957-4dd9-a44b-c55c200b7257 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:49.453687 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8f46b-a957-4dd9-a44b-c55c200b7257 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:49.461389 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 635/1277] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 12 14:03:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:03:49.467252 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f47e54-6735-4631-852d-602f96bb5d5d None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:49.573356 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f47e54-6735-4631-852d-602f96bb5d5d None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:49.605807 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae04e9-9df2-44ff-83d7-9a4feac7b22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:49.606424 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae04e9-9df2-44ff-83d7-9a4feac7b22d None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:49.606702 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae04e9-9df2-44ff-83d7-9a4feac7b22d None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:49.607005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae04e9-9df2-44ff-83d7-9a4feac7b22d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:49.617139 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae04e9-9df2-44ff-83d7-9a4feac7b22d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:49.618549 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae04e9-9df2-44ff-83d7-9a4feac7b22d None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:49.632610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-86ae04e9-9df2-44ff-83d7-9a4feac7b22d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:03:49.632610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:03:49.632610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:03:49.632610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:03:49.632610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:03:49.632610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:03:49.632610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:03:49.632610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:03:49.632610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:03:49.632610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:03:49.632610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:03:49.632610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:03:49.632610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:03:49.632610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:03:49.632610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:03:49.632610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:03:49.633755 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:03:49.633755 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:03:49.633755 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:03:49.633755 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:03:49.633755 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:03:49.633755 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:03:49.633755 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:03:49.633755 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:03:49.633755 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:03:49.633755 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:03:49.633755 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:03:49.633755 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:03:49.633755 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:03:49.633755 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:03:49.633755 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:03:49.633755 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:03:49.633755 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:03:49.635080 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:03:49.635080 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:03:49.635080 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:03:49.635080 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 643/1278] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:03:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:03:49.639520 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8f46b-a957-4dd9-a44b-c55c200b7257 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:49.735284 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8f46b-a957-4dd9-a44b-c55c200b7257 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:49.767889 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a64eca-29d4-4d9a-a112-9f009d6298d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:49.768474 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a64eca-29d4-4d9a-a112-9f009d6298d3 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:49.768747 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a64eca-29d4-4d9a-a112-9f009d6298d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:49.769013 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a64eca-29d4-4d9a-a112-9f009d6298d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:49.769639 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a64eca-29d4-4d9a-a112-9f009d6298d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:49.769932 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a64eca-29d4-4d9a-a112-9f009d6298d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:49.771306 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a64eca-29d4-4d9a-a112-9f009d6298d3 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:49.782493 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 636/1279] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:03:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:03:49.788573 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae04e9-9df2-44ff-83d7-9a4feac7b22d None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:49.818633 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:49.885859 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae04e9-9df2-44ff-83d7-9a4feac7b22d None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:49.919172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e753311-109b-4c32-ba6d-0d67a180d1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:49.919783 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e753311-109b-4c32-ba6d-0d67a180d1dc None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:49.920057 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e753311-109b-4c32-ba6d-0d67a180d1dc None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:49.920322 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e753311-109b-4c32-ba6d-0d67a180d1dc None admin] PATH_INFO: `/v3/users/placement` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:49.927386 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e753311-109b-4c32-ba6d-0d67a180d1dc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:49.928778 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e753311-109b-4c32-ba6d-0d67a180d1dc None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:49.939581 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-3e753311-109b-4c32-ba6d-0d67a180d1dc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 12 14:03:49.939581 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:03:49.939581 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:03:49.939581 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:03:49.939581 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:03:49.939581 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:03:49.939581 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:03:49.939581 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:03:49.939581 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:03:49.939581 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:03:49.939581 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:03:49.939581 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:03:49.939581 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 14:03:49.939581 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 14:03:49.939581 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 14:03:49.939581 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 14:03:49.940734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:03:49.940734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:03:49.940734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 14:03:49.940734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:03:49.940734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 14:03:49.940734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 14:03:49.940734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:03:49.940734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:03:49.940734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:03:49.940734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:03:49.940734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:03:49.940734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:03:49.940734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:03:49.940734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:03:49.940734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:03:49.940734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:03:49.940734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:03:49.940734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:03:49.941704 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:03:49.941704 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:03:49.941704 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 14:03:49.941704 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 14:03:49.941704 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 14:03:49.941704 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 14:03:49.941704 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 14:03:49.941704 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 14:03:49.941704 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 12 14:03:49.941704 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:03:49.941704 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 644/1280] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:03:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:03:49.946704 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a64eca-29d4-4d9a-a112-9f009d6298d3 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:49.950421 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:50.043242 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a64eca-29d4-4d9a-a112-9f009d6298d3 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:50.075563 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb7e973-fae3-4d34-a714-324ce46b77c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:50.076155 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7e973-fae3-4d34-a714-324ce46b77c4 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:50.076434 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7e973-fae3-4d34-a714-324ce46b77c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:50.076704 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7e973-fae3-4d34-a714-324ce46b77c4 None admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:50.077421 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb7e973-fae3-4d34-a714-324ce46b77c4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:50.077724 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb7e973-fae3-4d34-a714-324ce46b77c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:50.079164 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb7e973-fae3-4d34-a714-324ce46b77c4 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:50.096451 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 637/1281] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Sep 12 14:03:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:03:50.102129 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e753311-109b-4c32-ba6d-0d67a180d1dc None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:50.189771 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:50.199595 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e753311-109b-4c32-ba6d-0d67a180d1dc None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:50.231924 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f851f99-bc29-4b03-a7d0-b9b508f5e2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:50.232522 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f851f99-bc29-4b03-a7d0-b9b508f5e2c6 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:50.232799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f851f99-bc29-4b03-a7d0-b9b508f5e2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:50.233067 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f851f99-bc29-4b03-a7d0-b9b508f5e2c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:50.296380 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f851f99-bc29-4b03-a7d0-b9b508f5e2c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:50.297738 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f851f99-bc29-4b03-a7d0-b9b508f5e2c6 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:50.380992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:50.390944 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 645/1282] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 12 14:03:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 14:03:50.397420 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb7e973-fae3-4d34-a714-324ce46b77c4 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:50.455581 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:50.492283 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb7e973-fae3-4d34-a714-324ce46b77c4 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:50.525747 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae7856-788b-4d2c-9456-9fa472c49d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:50.526352 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae7856-788b-4d2c-9456-9fa472c49d14 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:50.526625 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae7856-788b-4d2c-9456-9fa472c49d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:50.526917 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae7856-788b-4d2c-9456-9fa472c49d14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:50.536918 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae7856-788b-4d2c-9456-9fa472c49d14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:50.538327 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae7856-788b-4d2c-9456-9fa472c49d14 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:50.552910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-97ae7856-788b-4d2c-9456-9fa472c49d14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:03:50.552910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:03:50.552910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:03:50.552910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:03:50.552910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:03:50.552910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:03:50.552910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:03:50.552910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:03:50.552910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:03:50.552910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:03:50.552910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:03:50.552910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:03:50.552910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 14:03:50.552910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 14:03:50.552910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 14:03:50.552910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 14:03:50.553980 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:03:50.553980 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:03:50.553980 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:03:50.553980 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:03:50.553980 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:03:50.553980 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:03:50.553980 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:03:50.553980 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:03:50.553980 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:03:50.553980 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:03:50.553980 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:03:50.553980 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:03:50.553980 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:03:50.553980 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:03:50.553980 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:03:50.553980 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:03:50.553980 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 14:03:50.554940 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 14:03:50.554940 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 14:03:50.554940 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:03:50.554940 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 638/1283] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:03:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:03:50.559762 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f851f99-bc29-4b03-a7d0-b9b508f5e2c6 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:50.660273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f851f99-bc29-4b03-a7d0-b9b508f5e2c6 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:50.692005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a66fc-729e-419b-8756-795e637520ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:50.692597 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a66fc-729e-419b-8756-795e637520ad None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:50.692877 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a66fc-729e-419b-8756-795e637520ad None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:50.693148 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a66fc-729e-419b-8756-795e637520ad None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:50.693777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077a66fc-729e-419b-8756-795e637520ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:50.694074 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077a66fc-729e-419b-8756-795e637520ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:50.695709 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077a66fc-729e-419b-8756-795e637520ad None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:50.707155 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 646/1284] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 12 14:03:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:03:50.712784 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae7856-788b-4d2c-9456-9fa472c49d14 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:50.808203 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae7856-788b-4d2c-9456-9fa472c49d14 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:50.821351 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:50.841595 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3488ae-6373-4281-a777-b492bfdc34c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:50.842217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3488ae-6373-4281-a777-b492bfdc34c4 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:50.842493 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3488ae-6373-4281-a777-b492bfdc34c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:50.842767 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3488ae-6373-4281-a777-b492bfdc34c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:50.849687 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3488ae-6373-4281-a777-b492bfdc34c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:50.851130 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3488ae-6373-4281-a777-b492bfdc34c4 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:50.862876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application [None req-7e3488ae-6373-4281-a777-b492bfdc34c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:03:50.862876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:03:50.862876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:03:50.862876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:03:50.862876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:03:50.862876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:03:50.862876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:03:50.862876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:03:50.862876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:03:50.862876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:03:50.862876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:03:50.862876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:03:50.862876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 14:03:50.862876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 14:03:50.862876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 14:03:50.863896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 14:03:50.863896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:03:50.863896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:03:50.863896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:03:50.863896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:03:50.863896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:03:50.863896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:03:50.863896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:03:50.863896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application with Lock( Sep 12 14:03:50.863896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:03:50.863896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:03:50.863896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:03:50.863896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:03:50.863896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:03:50.863896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:03:50.863896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:03:50.863896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:03:50.864949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 14:03:50.864949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 14:03:50.864949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 14:03:50.864949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 14:03:50.864949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 14:03:50.864949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 14:03:50.864949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 14:03:50.864949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: ERROR keystone.server.flask.application Sep 12 14:03:50.864949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 639/1285] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 12 14:03:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:03:50.869734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a66fc-729e-419b-8756-795e637520ad None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:50.919395 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:50.971399 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a66fc-729e-419b-8756-795e637520ad None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:51.002780 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e068b54-211a-46f1-a570-d09230235d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:51.003404 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e068b54-211a-46f1-a570-d09230235d70 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:51.003679 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e068b54-211a-46f1-a570-d09230235d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:51.003946 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e068b54-211a-46f1-a570-d09230235d70 None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:51.004624 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e068b54-211a-46f1-a570-d09230235d70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:51.004926 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e068b54-211a-46f1-a570-d09230235d70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:51.006319 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e068b54-211a-46f1-a570-d09230235d70 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:51.018171 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 647/1286] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 12 14:03:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:03:51.024447 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3488ae-6373-4281-a777-b492bfdc34c4 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:51.120607 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3488ae-6373-4281-a777-b492bfdc34c4 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:51.153314 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29387b9-51cc-43ab-85fc-c953e52e51ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:51.153894 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29387b9-51cc-43ab-85fc-c953e52e51ef None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:51.154168 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29387b9-51cc-43ab-85fc-c953e52e51ef None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:51.154435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29387b9-51cc-43ab-85fc-c953e52e51ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:51.155170 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29387b9-51cc-43ab-85fc-c953e52e51ef None admin] RBAC: Adding query filter params (role.id=e14564f00daf48a28d4e32da8cd2efad, scope.project.id=f666a617f8d14eb18bbeb076032b82f2, user.id=ce508bb9aa164a7fb1831a9b5ff5b932) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:51.155469 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29387b9-51cc-43ab-85fc-c953e52e51ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:51.156882 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29387b9-51cc-43ab-85fc-c953e52e51ef None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:51.162491 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 640/1287] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 12 14:03:51 2022] GET /identity/v3/role_assignments?user.id=ce508bb9aa164a7fb1831a9b5ff5b932&scope.project.id=f666a617f8d14eb18bbeb076032b82f2&role.id=e14564f00daf48a28d4e32da8cd2efad => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:03:53.302283 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a337f1f-7f94-4406-bf9e-8755b09df6a4 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:53.302790 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a337f1f-7f94-4406-bf9e-8755b09df6a4 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:53.302923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a337f1f-7f94-4406-bf9e-8755b09df6a4 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:53.304167 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 648/1288] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:03:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:03:53.309354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418ae77-e255-4809-b294-43d0b6bbcb9f None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:53.310061 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418ae77-e255-4809-b294-43d0b6bbcb9f None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:53.310331 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418ae77-e255-4809-b294-43d0b6bbcb9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:53.328669 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-5418ae77-e255-4809-b294-43d0b6bbcb9f None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:03:53.330430 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-5418ae77-e255-4809-b294-43d0b6bbcb9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:03:53.358120 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:53.394245 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 641/1289] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:03:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:03:53.666157 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a337f1f-7f94-4406-bf9e-8755b09df6a4 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:53.762695 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a337f1f-7f94-4406-bf9e-8755b09df6a4 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:53.772713 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:53.794830 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17761a0-7add-47b9-bc9e-eea6feabb0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:53.795462 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17761a0-7add-47b9-bc9e-eea6feabb0fc None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:53.795736 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17761a0-7add-47b9-bc9e-eea6feabb0fc None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:53.796002 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17761a0-7add-47b9-bc9e-eea6feabb0fc None admin] PATH_INFO: `/v3/services/placement` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:53.796578 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17761a0-7add-47b9-bc9e-eea6feabb0fc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:53.797905 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17761a0-7add-47b9-bc9e-eea6feabb0fc None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:53.806897 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-a17761a0-7add-47b9-bc9e-eea6feabb0fc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 14:03:53.806897 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:03:53.806897 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 14:03:53.806897 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 14:03:53.806897 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 14:03:53.806897 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 14:03:53.806897 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 14:03:53.806897 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:03:53.806897 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 14:03:53.806897 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:03:53.806897 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 14:03:53.806897 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:03:53.806897 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:03:53.806897 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:03:53.806897 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:03:53.806897 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:03:53.806897 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:03:53.808123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:03:53.808123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:03:53.808123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:03:53.808123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:03:53.808123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:03:53.808123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:03:53.808123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 14:03:53.808123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 14:03:53.808123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 14:03:53.808123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 14:03:53.808123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:03:53.808123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:03:53.808123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:03:53.808123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:03:53.808123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:03:53.808123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:03:53.809108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:03:53.809108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:03:53.809108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:03:53.809108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:03:53.809108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:03:53.809108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:03:53.809108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:03:53.809108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:03:53.809108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:03:53.809108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:03:53.809108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 14:03:53.809108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:03:53.809108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 14:03:53.809108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:03:53.809108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 649/1290] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Sep 12 14:03:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 142 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:03:53.813758 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5418ae77-e255-4809-b294-43d0b6bbcb9f None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:53.912341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5418ae77-e255-4809-b294-43d0b6bbcb9f None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:53.945794 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215448f-268c-46ea-9f6f-cfacd512453c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:53.946325 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215448f-268c-46ea-9f6f-cfacd512453c None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:53.946602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215448f-268c-46ea-9f6f-cfacd512453c None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:53.946895 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215448f-268c-46ea-9f6f-cfacd512453c None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:53.947519 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e215448f-268c-46ea-9f6f-cfacd512453c None admin] RBAC: Adding query filter params (name=placement) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:53.947820 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e215448f-268c-46ea-9f6f-cfacd512453c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:53.949181 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e215448f-268c-46ea-9f6f-cfacd512453c None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:53.953789 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 642/1291] 10.222.0.42 () {56 vars in 1116 bytes} [Mon Sep 12 14:03:53 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:03:53.959525 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17761a0-7add-47b9-bc9e-eea6feabb0fc None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:54.055497 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17761a0-7add-47b9-bc9e-eea6feabb0fc None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:54.087014 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f4bd9-8062-41d0-bcda-3c6f826a861f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:54.087594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f4bd9-8062-41d0-bcda-3c6f826a861f None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:54.087869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f4bd9-8062-41d0-bcda-3c6f826a861f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:54.088135 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f4bd9-8062-41d0-bcda-3c6f826a861f None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:54.088750 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f4bd9-8062-41d0-bcda-3c6f826a861f None admin] RBAC: Adding query filter params (type=placement) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:54.089044 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f4bd9-8062-41d0-bcda-3c6f826a861f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:54.090368 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f4bd9-8062-41d0-bcda-3c6f826a861f None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:54.094540 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 650/1292] 10.222.0.42 () {56 vars in 1116 bytes} [Mon Sep 12 14:03:53 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:03:56.209679 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1c9a5-c9ea-4d17-82fd-5554292eb2e8 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:56.210186 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1c9a5-c9ea-4d17-82fd-5554292eb2e8 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:56.210186 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1c9a5-c9ea-4d17-82fd-5554292eb2e8 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:56.211555 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 643/1293] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:03:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:03:56.216762 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a4931-e53f-4287-b96e-71bb0d563ef3 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:56.217139 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a4931-e53f-4287-b96e-71bb0d563ef3 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:56.217316 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a4931-e53f-4287-b96e-71bb0d563ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:56.235110 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-815a4931-e53f-4287-b96e-71bb0d563ef3 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:03:56.236830 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-815a4931-e53f-4287-b96e-71bb0d563ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:03:56.265291 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:56.301818 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 651/1294] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:03:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:03:56.568835 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1c9a5-c9ea-4d17-82fd-5554292eb2e8 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:56.599457 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:56.667485 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1c9a5-c9ea-4d17-82fd-5554292eb2e8 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:56.699343 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c10e8-39dc-4e64-a5ff-a5a6eb642dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:56.699930 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c10e8-39dc-4e64-a5ff-a5a6eb642dbf None admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:56.700204 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c10e8-39dc-4e64-a5ff-a5a6eb642dbf None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:56.700472 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c10e8-39dc-4e64-a5ff-a5a6eb642dbf None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:56.701119 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519c10e8-39dc-4e64-a5ff-a5a6eb642dbf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:56.702458 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519c10e8-39dc-4e64-a5ff-a5a6eb642dbf None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:56.716287 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 644/1295] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Sep 12 14:03:56 2022] POST /identity/v3/services => generated 246 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:03:58.845904 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74bb15-efa8-4d1c-b94c-9b5d1781b795 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:58.846312 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74bb15-efa8-4d1c-b94c-9b5d1781b795 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:58.846455 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74bb15-efa8-4d1c-b94c-9b5d1781b795 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:58.847851 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 652/1296] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:03:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:03:58.853047 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e03ee-1be5-4a6a-9b9d-483d0b92bf2e None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:58.853410 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e03ee-1be5-4a6a-9b9d-483d0b92bf2e None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:58.853601 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e03ee-1be5-4a6a-9b9d-483d0b92bf2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:58.871135 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-430e03ee-1be5-4a6a-9b9d-483d0b92bf2e None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:03:58.872307 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-430e03ee-1be5-4a6a-9b9d-483d0b92bf2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:03:58.942202 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 645/1297] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:03:58 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:03:59.206027 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf74bb15-efa8-4d1c-b94c-9b5d1781b795 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:59.303294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf74bb15-efa8-4d1c-b94c-9b5d1781b795 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:59.335073 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251d9ea-0a4f-4ffd-8793-a8b832c85019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:59.335688 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251d9ea-0a4f-4ffd-8793-a8b832c85019 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:59.335966 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251d9ea-0a4f-4ffd-8793-a8b832c85019 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:59.336236 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251d9ea-0a4f-4ffd-8793-a8b832c85019 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:59.336810 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251d9ea-0a4f-4ffd-8793-a8b832c85019 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:59.338151 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251d9ea-0a4f-4ffd-8793-a8b832c85019 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:59.347212 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-5251d9ea-0a4f-4ffd-8793-a8b832c85019 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 14:03:59.347212 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:03:59.347212 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 14:03:59.347212 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 14:03:59.347212 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 14:03:59.347212 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 14:03:59.347212 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 14:03:59.347212 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:03:59.347212 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 14:03:59.347212 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:03:59.347212 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 14:03:59.347212 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:03:59.347212 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:03:59.347212 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:03:59.347212 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:03:59.347212 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:03:59.347212 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:03:59.348397 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:03:59.348397 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:03:59.348397 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:03:59.348397 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:03:59.348397 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:03:59.348397 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:03:59.348397 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 14:03:59.348397 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 14:03:59.348397 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 14:03:59.348397 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 14:03:59.348397 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:03:59.348397 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:03:59.348397 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:03:59.348397 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:03:59.348397 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:03:59.348397 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:03:59.349482 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:03:59.349482 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:03:59.349482 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:03:59.349482 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:03:59.349482 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:03:59.349482 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:03:59.349482 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:03:59.349482 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:03:59.349482 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:03:59.349482 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:03:59.349482 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 14:03:59.349482 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:03:59.349482 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 14:03:59.349482 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:03:59.349482 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 653/1298] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Sep 12 14:03:59 2022] GET /identity/v3/services/placement => generated 90 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:03:59.354239 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e03ee-1be5-4a6a-9b9d-483d0b92bf2e None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:59.372301 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:59.449895 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e03ee-1be5-4a6a-9b9d-483d0b92bf2e None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:59.482771 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e9e6be-852a-426b-80c5-805dd73c74c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:59.483319 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9e6be-852a-426b-80c5-805dd73c74c5 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:59.483590 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9e6be-852a-426b-80c5-805dd73c74c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:59.483854 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9e6be-852a-426b-80c5-805dd73c74c5 None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:59.484460 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e9e6be-852a-426b-80c5-805dd73c74c5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:59.484749 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e9e6be-852a-426b-80c5-805dd73c74c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:59.486673 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e9e6be-852a-426b-80c5-805dd73c74c5 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:59.492386 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 646/1299] 10.222.0.42 () {56 vars in 1116 bytes} [Mon Sep 12 14:03:59 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:03:59.499544 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251d9ea-0a4f-4ffd-8793-a8b832c85019 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:59.536759 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:59.602216 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251d9ea-0a4f-4ffd-8793-a8b832c85019 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:59.634002 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d656d-dfe2-41f9-9002-fa88c89d1ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:59.634588 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d656d-dfe2-41f9-9002-fa88c89d1ecd None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:59.634888 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d656d-dfe2-41f9-9002-fa88c89d1ecd None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:59.635162 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d656d-dfe2-41f9-9002-fa88c89d1ecd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:59.635846 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d656d-dfe2-41f9-9002-fa88c89d1ecd None admin] RBAC: Adding query filter params (interface=public, service_id=aa553205ce494c39b8af230fed571240, region_id=RegionOne) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:03:59.636147 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d656d-dfe2-41f9-9002-fa88c89d1ecd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:59.637508 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d656d-dfe2-41f9-9002-fa88c89d1ecd None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:59.642809 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 654/1300] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Sep 12 14:03:59 2022] GET /identity/v3/endpoints?service_id=aa553205ce494c39b8af230fed571240&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:03:59.648425 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e9e6be-852a-426b-80c5-805dd73c74c5 None admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:03:59.726768 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:03:59.744451 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e9e6be-852a-426b-80c5-805dd73c74c5 None admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:03:59.776861 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349e019e-8d37-4dd4-8cd1-316b7f8045f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:03:59.777446 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e019e-8d37-4dd4-8cd1-316b7f8045f1 None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:03:59.777722 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e019e-8d37-4dd4-8cd1-316b7f8045f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:03:59.777989 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e019e-8d37-4dd4-8cd1-316b7f8045f1 None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:03:59.778577 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349e019e-8d37-4dd4-8cd1-316b7f8045f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:03:59.779952 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349e019e-8d37-4dd4-8cd1-316b7f8045f1 None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:03:59.787098 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 647/1301] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Sep 12 14:03:59 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:04:01.876231 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943218f5-0c5a-46d4-9bb1-87e04c8f7837 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:04:01.876702 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943218f5-0c5a-46d4-9bb1-87e04c8f7837 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:04:01.876814 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943218f5-0c5a-46d4-9bb1-87e04c8f7837 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:04:01.878100 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 655/1302] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:04:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:04:01.883143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7b50a-5822-446f-ad34-fbe726ef2bba None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:04:01.883525 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7b50a-5822-446f-ad34-fbe726ef2bba None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:04:01.883678 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7b50a-5822-446f-ad34-fbe726ef2bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:04:01.901571 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-3ab7b50a-5822-446f-ad34-fbe726ef2bba None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:04:01.902726 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-3ab7b50a-5822-446f-ad34-fbe726ef2bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:04:01.965973 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 648/1303] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:04:01 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 14:04:02.233103 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943218f5-0c5a-46d4-9bb1-87e04c8f7837 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:04:02.323600 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:04:02.330751 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943218f5-0c5a-46d4-9bb1-87e04c8f7837 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:04:02.362079 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89505d9d-c1e4-4e79-a7cd-0f1789433b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:04:02.362684 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89505d9d-c1e4-4e79-a7cd-0f1789433b7f None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:04:02.362987 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89505d9d-c1e4-4e79-a7cd-0f1789433b7f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:04:02.363253 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89505d9d-c1e4-4e79-a7cd-0f1789433b7f None admin] PATH_INFO: `/v3/services/placement` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:04:02.363818 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89505d9d-c1e4-4e79-a7cd-0f1789433b7f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:04:02.365146 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89505d9d-c1e4-4e79-a7cd-0f1789433b7f None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:04:02.373562 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-89505d9d-c1e4-4e79-a7cd-0f1789433b7f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 14:04:02.373562 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:04:02.373562 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 14:04:02.373562 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 14:04:02.373562 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 14:04:02.373562 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 14:04:02.373562 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 14:04:02.373562 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:04:02.373562 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 14:04:02.373562 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:04:02.373562 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 14:04:02.373562 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:04:02.373562 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 14:04:02.373562 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 14:04:02.373562 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 14:04:02.373562 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 14:04:02.373562 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 14:04:02.374719 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 14:04:02.374719 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 14:04:02.374719 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 14:04:02.374719 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 14:04:02.374719 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 14:04:02.374719 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 14:04:02.374719 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 14:04:02.374719 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 14:04:02.374719 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 14:04:02.374719 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 14:04:02.374719 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 14:04:02.374719 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 14:04:02.374719 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 14:04:02.374719 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 14:04:02.374719 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 14:04:02.374719 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 14:04:02.375768 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 14:04:02.375768 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 12 14:04:02.375768 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 14:04:02.375768 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 12 14:04:02.375768 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 14:04:02.375768 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 14:04:02.375768 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 14:04:02.375768 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 12 14:04:02.375768 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 14:04:02.375768 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 12 14:04:02.375768 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 14:04:02.375768 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 14:04:02.375768 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 14:04:02.375768 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 12 14:04:02.375768 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 656/1304] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Sep 12 14:04:02 2022] GET /identity/v3/services/placement => generated 90 bytes in 142 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 14:04:02.381271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7b50a-5822-446f-ad34-fbe726ef2bba None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:04:02.453957 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:04:02.476576 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7b50a-5822-446f-ad34-fbe726ef2bba None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:04:02.508781 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c490697-79d6-4b2a-bc9f-05c82ec557fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:04:02.509314 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c490697-79d6-4b2a-bc9f-05c82ec557fb None admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:04:02.509589 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c490697-79d6-4b2a-bc9f-05c82ec557fb None admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:04:02.509857 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c490697-79d6-4b2a-bc9f-05c82ec557fb None admin] PATH_INFO: `/v3/services` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:04:02.510471 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c490697-79d6-4b2a-bc9f-05c82ec557fb None admin] RBAC: Adding query filter params (name=placement) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:04:02.510768 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c490697-79d6-4b2a-bc9f-05c82ec557fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:04:02.512138 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c490697-79d6-4b2a-bc9f-05c82ec557fb None admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:04:02.517625 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 649/1305] 10.222.0.42 () {56 vars in 1116 bytes} [Mon Sep 12 14:04:02 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:04:02.523309 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89505d9d-c1e4-4e79-a7cd-0f1789433b7f None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:04:02.618017 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89505d9d-c1e4-4e79-a7cd-0f1789433b7f None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:04:02.650167 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b9cd5-e88b-47ea-997b-2e58b6ab80d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:04:02.650749 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b9cd5-e88b-47ea-997b-2e58b6ab80d6 None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:04:02.651052 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b9cd5-e88b-47ea-997b-2e58b6ab80d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:04:02.651324 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b9cd5-e88b-47ea-997b-2e58b6ab80d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:04:02.651987 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b9cd5-e88b-47ea-997b-2e58b6ab80d6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:04:02.653317 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b9cd5-e88b-47ea-997b-2e58b6ab80d6 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:04:02.671727 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 657/1306] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:04:02 2022] POST /identity/v3/endpoints => generated 331 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:04:12.514146 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=30 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:04:12.667486 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:05:06.316159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa21972-7626-4352-a8bb-e62a2ef66f66 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:06.316610 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa21972-7626-4352-a8bb-e62a2ef66f66 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:06.316878 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa21972-7626-4352-a8bb-e62a2ef66f66 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:06.318273 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 650/1307] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:05:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:05:06.333169 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee62d9-8792-4b24-acb8-030f4ed4f5fa None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:06.335896 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee62d9-8792-4b24-acb8-030f4ed4f5fa None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:06.337780 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee62d9-8792-4b24-acb8-030f4ed4f5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:06.520249 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-99ee62d9-8792-4b24-acb8-030f4ed4f5fa None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:05:06.521689 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-99ee62d9-8792-4b24-acb8-030f4ed4f5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:06.653037 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 658/1308] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:05:06.946102 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ba399-fb3f-4827-8c92-4669b7a2c842 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:06.946447 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ba399-fb3f-4827-8c92-4669b7a2c842 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:06.946819 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ba399-fb3f-4827-8c92-4669b7a2c842 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:06.948408 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 651/1309] 10.222.0.42 () {54 vars in 965 bytes} [Mon Sep 12 14:05:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:05:06.954812 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2b20b-cdb1-40d3-ad97-230c65709366 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:06.955147 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2b20b-cdb1-40d3-ad97-230c65709366 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:06.955419 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2b20b-cdb1-40d3-ad97-230c65709366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:07.039909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-1ed2b20b-cdb1-40d3-ad97-230c65709366 None None] MFA Rules not processed for user `865b1647468c4efe8efe4febe2203a58`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:05:07.041130 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1ed2b20b-cdb1-40d3-ad97-230c65709366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:07.066677 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:05:07.136401 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 659/1310] 10.222.0.42 () {58 vars in 1062 bytes} [Mon Sep 12 14:05:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:05:07.146306 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76ba399-fb3f-4827-8c92-4669b7a2c842 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:05:07.148212 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-c76ba399-fb3f-4827-8c92-4669b7a2c842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:07.272560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76ba399-fb3f-4827-8c92-4669b7a2c842 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:05:07.355385 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb64f52-94b4-4414-aeaa-d8cfb56931f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:05:07.356027 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb64f52-94b4-4414-aeaa-d8cfb56931f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:07.356330 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb64f52-94b4-4414-aeaa-d8cfb56931f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:07.356521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb64f52-94b4-4414-aeaa-d8cfb56931f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:07.359021 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-efb64f52-94b4-4414-aeaa-d8cfb56931f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:07.418070 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:05:07.477460 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb64f52-94b4-4414-aeaa-d8cfb56931f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:05:07.478805 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb64f52-94b4-4414-aeaa-d8cfb56931f1 service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:05:07.499113 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 652/1311] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:05:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:05:09.985369 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8bddc2-f3f4-4c96-8f62-2baf8afe56ac None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:09.985889 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8bddc2-f3f4-4c96-8f62-2baf8afe56ac None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:09.985889 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8bddc2-f3f4-4c96-8f62-2baf8afe56ac None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:09.987326 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 660/1312] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:05:09.992593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d9a46-e506-40ba-95ce-d7f700425033 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:09.992980 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d9a46-e506-40ba-95ce-d7f700425033 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:09.993135 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d9a46-e506-40ba-95ce-d7f700425033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:10.062782 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-274d9a46-e506-40ba-95ce-d7f700425033 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:05:10.064005 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-274d9a46-e506-40ba-95ce-d7f700425033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:10.165193 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 653/1313] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:05:10.464357 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8bddc2-f3f4-4c96-8f62-2baf8afe56ac None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:05:10.465937 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1f8bddc2-f3f4-4c96-8f62-2baf8afe56ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:10.568893 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8bddc2-f3f4-4c96-8f62-2baf8afe56ac None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:05:10.639933 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:05:10.642275 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90138ba-f0e6-4e1e-bb45-9fc6fef6ed70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:05:10.643091 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90138ba-f0e6-4e1e-bb45-9fc6fef6ed70 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:10.643532 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90138ba-f0e6-4e1e-bb45-9fc6fef6ed70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:10.644000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90138ba-f0e6-4e1e-bb45-9fc6fef6ed70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:10.645640 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e90138ba-f0e6-4e1e-bb45-9fc6fef6ed70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:10.753702 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90138ba-f0e6-4e1e-bb45-9fc6fef6ed70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:05:10.755070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90138ba-f0e6-4e1e-bb45-9fc6fef6ed70 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:05:10.772936 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 661/1314] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:05:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:05:13.233681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea701262-92fa-44a0-addf-aef272684756 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:13.234195 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea701262-92fa-44a0-addf-aef272684756 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:13.234195 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea701262-92fa-44a0-addf-aef272684756 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:13.235511 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 654/1315] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:05:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:05:13.241421 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12f83b-c5a4-4dbe-b84b-36143e3e8f4c None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:13.241878 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12f83b-c5a4-4dbe-b84b-36143e3e8f4c None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:13.241974 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12f83b-c5a4-4dbe-b84b-36143e3e8f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:13.307152 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-2c12f83b-c5a4-4dbe-b84b-36143e3e8f4c None None] MFA Rules not processed for user `96b89bc850f04d96bedaea747c2c46b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:05:13.308368 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-2c12f83b-c5a4-4dbe-b84b-36143e3e8f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:13.359588 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:05:13.403012 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 662/1316] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:05:13 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:05:13.691498 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea701262-92fa-44a0-addf-aef272684756 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:05:13.693024 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-ea701262-92fa-44a0-addf-aef272684756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:13.702995 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:05:13.794095 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea701262-92fa-44a0-addf-aef272684756 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:05:13.869317 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd6b57-90f4-4dd7-93ce-924a7461fb01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:05:13.869913 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd6b57-90f4-4dd7-93ce-924a7461fb01 service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:13.870187 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd6b57-90f4-4dd7-93ce-924a7461fb01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:13.870454 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd6b57-90f4-4dd7-93ce-924a7461fb01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:13.872064 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-fcdd6b57-90f4-4dd7-93ce-924a7461fb01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:13.933114 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:05:13.978879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdd6b57-90f4-4dd7-93ce-924a7461fb01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:05:13.980236 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdd6b57-90f4-4dd7-93ce-924a7461fb01 service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:05:13.997725 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 655/1317] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:05:13 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:05:16.687054 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00171f-34d6-45fa-82d8-16d1d4a9dd66 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:16.687595 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00171f-34d6-45fa-82d8-16d1d4a9dd66 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:16.687693 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00171f-34d6-45fa-82d8-16d1d4a9dd66 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:16.688995 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 663/1318] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:05:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:05:16.695434 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7a085-5210-4755-ba61-cdd350603ca5 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:16.695876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7a085-5210-4755-ba61-cdd350603ca5 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:16.695972 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7a085-5210-4755-ba61-cdd350603ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:16.768038 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-09d7a085-5210-4755-ba61-cdd350603ca5 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:05:16.769345 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-09d7a085-5210-4755-ba61-cdd350603ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:16.871035 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 656/1319] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:05:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:05:17.163582 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e00171f-34d6-45fa-82d8-16d1d4a9dd66 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:05:17.165135 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7e00171f-34d6-45fa-82d8-16d1d4a9dd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:17.266964 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e00171f-34d6-45fa-82d8-16d1d4a9dd66 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:05:17.337620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e413d3-fdc8-4f63-afdb-55b8c93462e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:05:17.338209 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e413d3-fdc8-4f63-afdb-55b8c93462e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:17.338483 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e413d3-fdc8-4f63-afdb-55b8c93462e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:17.338752 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e413d3-fdc8-4f63-afdb-55b8c93462e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:17.340367 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-81e413d3-fdc8-4f63-afdb-55b8c93462e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:17.346999 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:05:17.445217 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e413d3-fdc8-4f63-afdb-55b8c93462e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:05:17.446543 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e413d3-fdc8-4f63-afdb-55b8c93462e3 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:05:17.464885 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 664/1320] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:05:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:05:20.231759 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8856ed9f-b3b3-4fcf-bd1a-ac86aba43658 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:20.232810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8856ed9f-b3b3-4fcf-bd1a-ac86aba43658 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:20.232810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8856ed9f-b3b3-4fcf-bd1a-ac86aba43658 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:20.234078 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 657/1321] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:05:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:05:20.239447 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ecef1-1be9-471b-96d2-ce0a31923291 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:20.239867 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ecef1-1be9-471b-96d2-ce0a31923291 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:20.239997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ecef1-1be9-471b-96d2-ce0a31923291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:20.314797 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-a61ecef1-1be9-471b-96d2-ce0a31923291 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:05:20.316063 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-a61ecef1-1be9-471b-96d2-ce0a31923291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:20.378461 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:05:20.421039 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 665/1322] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:05:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:05:20.711063 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8856ed9f-b3b3-4fcf-bd1a-ac86aba43658 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:05:20.712657 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-8856ed9f-b3b3-4fcf-bd1a-ac86aba43658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:20.732690 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:05:20.817148 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8856ed9f-b3b3-4fcf-bd1a-ac86aba43658 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:05:20.901573 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0cedd-a9e3-427b-96fd-677d02fff53c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:05:20.902176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0cedd-a9e3-427b-96fd-677d02fff53c service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:20.902450 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0cedd-a9e3-427b-96fd-677d02fff53c service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:20.902715 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0cedd-a9e3-427b-96fd-677d02fff53c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:20.904920 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-95b0cedd-a9e3-427b-96fd-677d02fff53c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:20.979107 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:05:21.017171 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b0cedd-a9e3-427b-96fd-677d02fff53c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:05:21.018603 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b0cedd-a9e3-427b-96fd-677d02fff53c service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:05:21.038050 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 658/1323] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:05:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:05:23.250775 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205948f-cd1e-4bf0-93d5-999c9bb3758a None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:23.251332 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205948f-cd1e-4bf0-93d5-999c9bb3758a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:23.251332 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205948f-cd1e-4bf0-93d5-999c9bb3758a None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:23.252637 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 666/1324] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:05:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:05:23.257936 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f16c5-ce7b-4688-9866-f7aabc80fc79 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:23.258376 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f16c5-ce7b-4688-9866-f7aabc80fc79 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:23.258600 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f16c5-ce7b-4688-9866-f7aabc80fc79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:23.328895 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-0b0f16c5-ce7b-4688-9866-f7aabc80fc79 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:05:23.331026 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-0b0f16c5-ce7b-4688-9866-f7aabc80fc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:23.431488 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 659/1325] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:05:23.715129 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205948f-cd1e-4bf0-93d5-999c9bb3758a None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:05:23.716686 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-5205948f-cd1e-4bf0-93d5-999c9bb3758a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:23.818740 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205948f-cd1e-4bf0-93d5-999c9bb3758a None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:05:23.893337 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8b710-0cdc-4842-9b8e-99fbaa886347 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:05:23.893941 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8b710-0cdc-4842-9b8e-99fbaa886347 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:23.894217 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8b710-0cdc-4842-9b8e-99fbaa886347 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:23.894486 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8b710-0cdc-4842-9b8e-99fbaa886347 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:23.896138 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b2d8b710-0cdc-4842-9b8e-99fbaa886347 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:23.912484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:05:24.002774 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8b710-0cdc-4842-9b8e-99fbaa886347 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:05:24.004295 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8b710-0cdc-4842-9b8e-99fbaa886347 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:05:24.021764 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 667/1326] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:05:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:05:26.228331 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aed516-3e08-4f68-ae79-d848fdc8b665 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:26.228857 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aed516-3e08-4f68-ae79-d848fdc8b665 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:26.228857 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aed516-3e08-4f68-ae79-d848fdc8b665 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:26.230120 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 660/1327] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:05:26.235803 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e9867-6bed-4647-885c-dadf5dcbf7db None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:26.236241 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e9867-6bed-4647-885c-dadf5dcbf7db None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:26.236366 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e9867-6bed-4647-885c-dadf5dcbf7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:26.302742 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-420e9867-6bed-4647-885c-dadf5dcbf7db None None] MFA Rules not processed for user `96b89bc850f04d96bedaea747c2c46b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:05:26.304008 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-420e9867-6bed-4647-885c-dadf5dcbf7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:26.368959 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:05:26.399735 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 668/1328] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:05:26 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:05:26.699791 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470e687-78f6-4586-b8e3-cb25c6bc8a1c None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:26.700067 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470e687-78f6-4586-b8e3-cb25c6bc8a1c None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:26.700339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470e687-78f6-4586-b8e3-cb25c6bc8a1c None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:26.702230 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 661/1329] 10.222.0.42 () {54 vars in 965 bytes} [Mon Sep 12 14:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:05:26.708580 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8237e2-1e54-4ecc-b78d-56aa2f39b058 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:26.708952 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8237e2-1e54-4ecc-b78d-56aa2f39b058 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:26.709311 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8237e2-1e54-4ecc-b78d-56aa2f39b058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:26.791410 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-1b8237e2-1e54-4ecc-b78d-56aa2f39b058 None None] MFA Rules not processed for user `865b1647468c4efe8efe4febe2203a58`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:05:26.792658 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1b8237e2-1e54-4ecc-b78d-56aa2f39b058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:26.873471 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 669/1330] 10.222.0.42 () {58 vars in 1062 bytes} [Mon Sep 12 14:05:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:05:26.882556 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470e687-78f6-4586-b8e3-cb25c6bc8a1c None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:05:26.884109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-b470e687-78f6-4586-b8e3-cb25c6bc8a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:26.915804 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:05:26.988605 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470e687-78f6-4586-b8e3-cb25c6bc8a1c None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:05:27.061865 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ceff0-4803-41f5-bdc3-ed34f32b61ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:05:27.062402 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ceff0-4803-41f5-bdc3-ed34f32b61ae service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:27.062673 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ceff0-4803-41f5-bdc3-ed34f32b61ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:27.062976 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ceff0-4803-41f5-bdc3-ed34f32b61ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:27.064550 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-413ceff0-4803-41f5-bdc3-ed34f32b61ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:27.141601 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:05:27.173020 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ceff0-4803-41f5-bdc3-ed34f32b61ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:05:27.174450 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ceff0-4803-41f5-bdc3-ed34f32b61ae service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:05:27.193127 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 662/1331] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:05:26 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:05:29.953251 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b61cd-5ad8-48ca-954a-4039dc871324 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:29.953809 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b61cd-5ad8-48ca-954a-4039dc871324 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:29.953809 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b61cd-5ad8-48ca-954a-4039dc871324 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:29.955155 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 670/1332] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:05:29.960284 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8417548-77a0-4362-a34f-8e9b8ac27f7a None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:29.960692 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8417548-77a0-4362-a34f-8e9b8ac27f7a None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:29.960830 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8417548-77a0-4362-a34f-8e9b8ac27f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:30.024872 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-f8417548-77a0-4362-a34f-8e9b8ac27f7a None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:05:30.026052 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-f8417548-77a0-4362-a34f-8e9b8ac27f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:30.119811 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 663/1333] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:05:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:05:30.397944 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b61cd-5ad8-48ca-954a-4039dc871324 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:05:30.399513 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-3d7b61cd-5ad8-48ca-954a-4039dc871324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:30.467227 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:05:30.500812 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b61cd-5ad8-48ca-954a-4039dc871324 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:05:30.572550 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7002be76-a7b9-4124-aa3c-a39ed3491a3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:05:30.573164 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002be76-a7b9-4124-aa3c-a39ed3491a3e service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:30.573429 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002be76-a7b9-4124-aa3c-a39ed3491a3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:30.573690 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002be76-a7b9-4124-aa3c-a39ed3491a3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:30.575294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7002be76-a7b9-4124-aa3c-a39ed3491a3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:30.681624 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7002be76-a7b9-4124-aa3c-a39ed3491a3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:05:30.683000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7002be76-a7b9-4124-aa3c-a39ed3491a3e service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:05:30.693156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:05:30.702189 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 671/1334] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:05:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:05:32.883575 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d35d1-27ce-4d82-8a87-40566cf06438 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:32.884063 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d35d1-27ce-4d82-8a87-40566cf06438 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:32.884155 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d35d1-27ce-4d82-8a87-40566cf06438 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:32.885489 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 664/1335] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:05:32.891560 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64b702-9de4-4ab7-8491-f5339f2af284 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:32.891992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64b702-9de4-4ab7-8491-f5339f2af284 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:32.892112 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64b702-9de4-4ab7-8491-f5339f2af284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:32.957717 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-ea64b702-9de4-4ab7-8491-f5339f2af284 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:05:32.958938 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ea64b702-9de4-4ab7-8491-f5339f2af284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:33.055735 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 672/1336] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:05:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:05:33.338521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6d35d1-27ce-4d82-8a87-40566cf06438 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:05:33.340109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-0f6d35d1-27ce-4d82-8a87-40566cf06438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:33.380243 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:05:33.442717 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6d35d1-27ce-4d82-8a87-40566cf06438 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:05:33.515727 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8ebd82-82ca-4386-94af-36cf75f9560a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:05:33.516321 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ebd82-82ca-4386-94af-36cf75f9560a service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:33.516597 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ebd82-82ca-4386-94af-36cf75f9560a service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:33.516865 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ebd82-82ca-4386-94af-36cf75f9560a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:33.518444 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-0c8ebd82-82ca-4386-94af-36cf75f9560a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:33.602316 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:05:33.627161 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8ebd82-82ca-4386-94af-36cf75f9560a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:05:33.628584 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8ebd82-82ca-4386-94af-36cf75f9560a service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:05:33.646931 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 665/1337] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:05:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:05:36.123740 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb53b5-7d43-4745-b64e-64ca49543e25 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:36.124266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb53b5-7d43-4745-b64e-64ca49543e25 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:36.124266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb53b5-7d43-4745-b64e-64ca49543e25 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:36.125585 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 673/1338] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:05:36.130858 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc93cd2-a536-471c-9ded-464ac0bf2837 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:36.131281 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc93cd2-a536-471c-9ded-464ac0bf2837 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:36.131429 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc93cd2-a536-471c-9ded-464ac0bf2837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:36.202236 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-2bc93cd2-a536-471c-9ded-464ac0bf2837 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:05:36.203466 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-2bc93cd2-a536-471c-9ded-464ac0bf2837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:36.299814 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 666/1339] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:05:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:05:36.584543 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb53b5-7d43-4745-b64e-64ca49543e25 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:05:36.586074 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b1cb53b5-7d43-4745-b64e-64ca49543e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:36.659915 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:05:36.687829 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb53b5-7d43-4745-b64e-64ca49543e25 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:05:36.758494 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91513e06-b734-42f4-974c-31936b4825ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:05:36.759116 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91513e06-b734-42f4-974c-31936b4825ff service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:36.759391 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91513e06-b734-42f4-974c-31936b4825ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:36.759656 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91513e06-b734-42f4-974c-31936b4825ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:36.761244 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-91513e06-b734-42f4-974c-31936b4825ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:36.867891 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91513e06-b734-42f4-974c-31936b4825ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:05:36.869308 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91513e06-b734-42f4-974c-31936b4825ff service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:05:36.886530 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 674/1340] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:05:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:05:41.053800 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c361f-792a-4d06-8f06-e0abc02c8f6b None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:41.054370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c361f-792a-4d06-8f06-e0abc02c8f6b None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:41.054370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c361f-792a-4d06-8f06-e0abc02c8f6b None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:41.055744 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 667/1341] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:05:41.061902 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1c193-fc72-4f0d-bbe3-74394fe36bd2 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:41.062310 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1c193-fc72-4f0d-bbe3-74394fe36bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:41.062529 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1c193-fc72-4f0d-bbe3-74394fe36bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:41.068329 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:05:41.129396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-f3f1c193-fc72-4f0d-bbe3-74394fe36bd2 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:05:41.130512 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f3f1c193-fc72-4f0d-bbe3-74394fe36bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:41.226334 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 675/1342] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:05:41.511079 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16c361f-792a-4d06-8f06-e0abc02c8f6b None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:05:41.512660 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-a16c361f-792a-4d06-8f06-e0abc02c8f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:41.560363 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:05:41.616725 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16c361f-792a-4d06-8f06-e0abc02c8f6b None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:05:41.691578 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0733e943-c98c-4699-980d-57a90548f4c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:05:41.692180 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733e943-c98c-4699-980d-57a90548f4c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:41.692454 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733e943-c98c-4699-980d-57a90548f4c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:41.692727 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733e943-c98c-4699-980d-57a90548f4c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:41.694569 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-0733e943-c98c-4699-980d-57a90548f4c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:41.791509 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:05:41.808087 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0733e943-c98c-4699-980d-57a90548f4c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:05:41.809558 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0733e943-c98c-4699-980d-57a90548f4c4 service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:05:41.830320 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 668/1343] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:05:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:05:44.632819 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7034c84-3bbf-486d-bdb5-83353577ff00 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:44.633359 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7034c84-3bbf-486d-bdb5-83353577ff00 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:44.633359 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7034c84-3bbf-486d-bdb5-83353577ff00 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:44.634741 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 676/1344] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:05:44.640773 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3e46b-8f14-4e2b-b947-6073b6c343ad None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:44.641179 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3e46b-8f14-4e2b-b947-6073b6c343ad None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:44.641311 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3e46b-8f14-4e2b-b947-6073b6c343ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:44.705676 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-62b3e46b-8f14-4e2b-b947-6073b6c343ad None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:05:44.706877 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-62b3e46b-8f14-4e2b-b947-6073b6c343ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:44.806310 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 669/1345] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:05:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:05:45.090359 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7034c84-3bbf-486d-bdb5-83353577ff00 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:05:45.092504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b7034c84-3bbf-486d-bdb5-83353577ff00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:45.187093 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:05:45.204136 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7034c84-3bbf-486d-bdb5-83353577ff00 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:05:45.275895 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec7d2c-4514-4731-824c-7e90a15de25b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:05:45.276502 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec7d2c-4514-4731-824c-7e90a15de25b service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:45.276775 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec7d2c-4514-4731-824c-7e90a15de25b service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:45.277041 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec7d2c-4514-4731-824c-7e90a15de25b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:45.279193 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-fdec7d2c-4514-4731-824c-7e90a15de25b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:45.389071 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec7d2c-4514-4731-824c-7e90a15de25b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:05:45.390520 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec7d2c-4514-4731-824c-7e90a15de25b service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:05:45.407882 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 677/1346] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:05:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:05:49.500961 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59bf94f-f81a-407c-b4ea-203dbc20454a None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:49.501488 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59bf94f-f81a-407c-b4ea-203dbc20454a None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:49.501488 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59bf94f-f81a-407c-b4ea-203dbc20454a None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:49.502877 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 670/1347] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:05:49.508520 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f5bf9-eca4-473a-b8d4-c40350d6fee9 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:49.508939 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f5bf9-eca4-473a-b8d4-c40350d6fee9 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:49.509066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f5bf9-eca4-473a-b8d4-c40350d6fee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:49.526161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:05:49.582090 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-5f5f5bf9-eca4-473a-b8d4-c40350d6fee9 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:05:49.583355 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-5f5f5bf9-eca4-473a-b8d4-c40350d6fee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:49.679421 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 678/1348] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:05:49.966152 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59bf94f-f81a-407c-b4ea-203dbc20454a None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:05:49.967626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-c59bf94f-f81a-407c-b4ea-203dbc20454a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:50.025180 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:05:50.073630 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59bf94f-f81a-407c-b4ea-203dbc20454a None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:05:50.149551 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0327e243-c176-482e-a9ce-acadec74b8c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:05:50.150153 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327e243-c176-482e-a9ce-acadec74b8c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:50.150429 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327e243-c176-482e-a9ce-acadec74b8c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:50.150699 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327e243-c176-482e-a9ce-acadec74b8c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:50.152347 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-0327e243-c176-482e-a9ce-acadec74b8c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:50.256178 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:05:50.260125 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0327e243-c176-482e-a9ce-acadec74b8c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:05:50.261552 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0327e243-c176-482e-a9ce-acadec74b8c3 service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:05:50.280751 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 671/1349] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:05:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:05:55.177000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9ce75-7095-435c-ae09-60a477b831d2 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:55.177453 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9ce75-7095-435c-ae09-60a477b831d2 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:55.177543 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9ce75-7095-435c-ae09-60a477b831d2 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:55.179079 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 679/1350] 10.222.0.42 () {54 vars in 921 bytes} [Mon Sep 12 14:05:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:05:55.185330 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387d85c-d434-40ea-9cd1-fe690f47ddb8 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:55.186163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387d85c-d434-40ea-9cd1-fe690f47ddb8 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:55.186633 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387d85c-d434-40ea-9cd1-fe690f47ddb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:55.272344 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-6387d85c-d434-40ea-9cd1-fe690f47ddb8 None None] MFA Rules not processed for user `ce508bb9aa164a7fb1831a9b5ff5b932`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:05:55.273755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-6387d85c-d434-40ea-9cd1-fe690f47ddb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:55.432240 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 672/1351] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Sep 12 14:05:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:05:59.646651 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b68e85-4c51-4f90-bb4b-3491f71025fd None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:59.647200 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b68e85-4c51-4f90-bb4b-3491f71025fd None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:59.647616 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b68e85-4c51-4f90-bb4b-3491f71025fd None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:59.649274 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 680/1352] 10.222.0.42 () {54 vars in 921 bytes} [Mon Sep 12 14:05:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:05:59.655060 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae2abd-7df7-4be9-9a8a-09a669f176bf None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:05:59.655932 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae2abd-7df7-4be9-9a8a-09a669f176bf None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:05:59.656448 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae2abd-7df7-4be9-9a8a-09a669f176bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:05:59.677748 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=59 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:05:59.724588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:05:59.748929 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-25ae2abd-7df7-4be9-9a8a-09a669f176bf None None] MFA Rules not processed for user `ce508bb9aa164a7fb1831a9b5ff5b932`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:05:59.750132 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-25ae2abd-7df7-4be9-9a8a-09a669f176bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:05:59.873956 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 673/1353] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Sep 12 14:05:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:06:01.157752 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0ba77-19f8-4f39-9c06-d0ceedfbf08d None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:01.158293 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0ba77-19f8-4f39-9c06-d0ceedfbf08d None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:01.158588 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0ba77-19f8-4f39-9c06-d0ceedfbf08d None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:01.159987 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 681/1354] 10.222.0.42 () {54 vars in 921 bytes} [Mon Sep 12 14:06:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:06:01.165034 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b5309-4557-45dc-9325-86e568fa2896 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:01.165511 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b5309-4557-45dc-9325-86e568fa2896 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:01.165623 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b5309-4557-45dc-9325-86e568fa2896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:01.242509 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-c74b5309-4557-45dc-9325-86e568fa2896 None None] MFA Rules not processed for user `ce508bb9aa164a7fb1831a9b5ff5b932`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:06:01.243733 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-c74b5309-4557-45dc-9325-86e568fa2896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:01.285069 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:01.350403 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 674/1355] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Sep 12 14:06:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:06:04.913473 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528db65a-50ef-4f80-99b4-61e867a330be None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:04.914013 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528db65a-50ef-4f80-99b4-61e867a330be None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:04.914013 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528db65a-50ef-4f80-99b4-61e867a330be None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:04.915367 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 682/1356] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:06:04.920829 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a14d4-52f6-4c58-9e24-b8f31779fe6b None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:04.921263 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a14d4-52f6-4c58-9e24-b8f31779fe6b None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:04.921355 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a14d4-52f6-4c58-9e24-b8f31779fe6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:04.993056 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-dc4a14d4-52f6-4c58-9e24-b8f31779fe6b None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:06:04.994238 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-dc4a14d4-52f6-4c58-9e24-b8f31779fe6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:05.095928 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 675/1357] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:06:05.400382 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97436bfd-ccef-4ee8-afd6-7e5743a11684 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:05.401034 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97436bfd-ccef-4ee8-afd6-7e5743a11684 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:05.401531 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97436bfd-ccef-4ee8-afd6-7e5743a11684 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:05.403127 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 683/1358] 10.222.0.42 () {54 vars in 962 bytes} [Mon Sep 12 14:06:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:06:05.416209 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb3888d-038b-42a4-aed3-c6e070941e02 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:05.416424 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb3888d-038b-42a4-aed3-c6e070941e02 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:05.416686 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb3888d-038b-42a4-aed3-c6e070941e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:05.437067 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:05.505611 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-1fb3888d-038b-42a4-aed3-c6e070941e02 None None] MFA Rules not processed for user `854ab2dcec5a48b18f409b33cb61b5da`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:06:05.506694 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-1fb3888d-038b-42a4-aed3-c6e070941e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:05.645854 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 676/1359] 10.222.0.42 () {58 vars in 1059 bytes} [Mon Sep 12 14:06:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:06:05.653843 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97436bfd-ccef-4ee8-afd6-7e5743a11684 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:06:05.655489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-97436bfd-ccef-4ee8-afd6-7e5743a11684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:05.815088 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97436bfd-ccef-4ee8-afd6-7e5743a11684 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:06:05.901799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:05.946072 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f90cd-5309-49aa-b866-53df0985f048 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:06:05.946683 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f90cd-5309-49aa-b866-53df0985f048 service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:05.946979 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f90cd-5309-49aa-b866-53df0985f048 service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:05.947252 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f90cd-5309-49aa-b866-53df0985f048 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:05.948942 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-938f90cd-5309-49aa-b866-53df0985f048 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:06.054047 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938f90cd-5309-49aa-b866-53df0985f048 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:06:06.055674 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938f90cd-5309-49aa-b866-53df0985f048 service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:06:06.075744 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 684/1360] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:06:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:06:08.488267 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45322ec7-b42c-4874-967d-3f626c814cf1 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:08.488267 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45322ec7-b42c-4874-967d-3f626c814cf1 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:08.489129 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45322ec7-b42c-4874-967d-3f626c814cf1 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:08.495194 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a27d8d-6e5b-4a26-bbdd-da7f3df95105 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:08.495806 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a27d8d-6e5b-4a26-bbdd-da7f3df95105 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:08.495806 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a27d8d-6e5b-4a26-bbdd-da7f3df95105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:08.499322 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 677/1361] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:06:08 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:06:08.567822 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-f3a27d8d-6e5b-4a26-bbdd-da7f3df95105 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:06:08.569621 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f3a27d8d-6e5b-4a26-bbdd-da7f3df95105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:08.615321 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:08.678989 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 685/1362] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:06:08.974773 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45322ec7-b42c-4874-967d-3f626c814cf1 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:06:08.977389 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-45322ec7-b42c-4874-967d-3f626c814cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:09.056252 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:09.161288 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45322ec7-b42c-4874-967d-3f626c814cf1 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:06:09.287264 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c1de52-19b4-4a96-9912-f83c0b63aeb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:06:09.287876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1de52-19b4-4a96-9912-f83c0b63aeb2 service nova] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:09.288148 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1de52-19b4-4a96-9912-f83c0b63aeb2 service nova] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:09.288413 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1de52-19b4-4a96-9912-f83c0b63aeb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:09.290003 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-44c1de52-19b4-4a96-9912-f83c0b63aeb2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:09.313145 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:09.382945 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c1de52-19b4-4a96-9912-f83c0b63aeb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:06:09.384529 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c1de52-19b4-4a96-9912-f83c0b63aeb2 service nova] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:06:09.403193 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 678/1363] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:06:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:06:11.877197 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d787fe-e3aa-409e-8c63-8b5ca9fb1215 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:11.877989 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d787fe-e3aa-409e-8c63-8b5ca9fb1215 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:11.878271 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d787fe-e3aa-409e-8c63-8b5ca9fb1215 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:11.879654 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 686/1364] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:06:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:06:11.884874 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a70ee-e09f-4c03-bf04-59a2db32f344 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:11.885306 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a70ee-e09f-4c03-bf04-59a2db32f344 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:11.885405 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a70ee-e09f-4c03-bf04-59a2db32f344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:11.954091 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-c41a70ee-e09f-4c03-bf04-59a2db32f344 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:06:11.955326 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-c41a70ee-e09f-4c03-bf04-59a2db32f344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:12.060841 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:12.067151 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 679/1365] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:06:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:06:12.351842 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48fa2ee-18d0-4964-8a8c-00bef285584a None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:12.352127 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48fa2ee-18d0-4964-8a8c-00bef285584a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:12.352429 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48fa2ee-18d0-4964-8a8c-00bef285584a None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:12.354067 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 687/1366] 10.222.0.42 () {54 vars in 962 bytes} [Mon Sep 12 14:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:06:12.360091 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c157e-d2e3-4a51-a454-ddb7dbc1f71b None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:12.360503 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c157e-d2e3-4a51-a454-ddb7dbc1f71b None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:12.360977 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c157e-d2e3-4a51-a454-ddb7dbc1f71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:12.439277 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-483c157e-d2e3-4a51-a454-ddb7dbc1f71b None None] MFA Rules not processed for user `854ab2dcec5a48b18f409b33cb61b5da`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:06:12.440608 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-483c157e-d2e3-4a51-a454-ddb7dbc1f71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:12.561340 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 680/1367] 10.222.0.42 () {58 vars in 1059 bytes} [Mon Sep 12 14:06:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:06:12.570140 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48fa2ee-18d0-4964-8a8c-00bef285584a None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:06:12.571804 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b48fa2ee-18d0-4964-8a8c-00bef285584a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:12.719996 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48fa2ee-18d0-4964-8a8c-00bef285584a None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:06:12.740009 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:12.845958 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144791df-ab58-4635-abb9-c38b9c6433bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:06:12.846563 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144791df-ab58-4635-abb9-c38b9c6433bb service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:12.846991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144791df-ab58-4635-abb9-c38b9c6433bb service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:12.847272 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144791df-ab58-4635-abb9-c38b9c6433bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:12.848909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-144791df-ab58-4635-abb9-c38b9c6433bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:12.944968 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144791df-ab58-4635-abb9-c38b9c6433bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:06:12.947465 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144791df-ab58-4635-abb9-c38b9c6433bb service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:06:12.967239 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 688/1368] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:06:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:06:13.100309 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d4d3f-4ecc-46b8-b81d-cd0da23a1789 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:13.101223 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d4d3f-4ecc-46b8-b81d-cd0da23a1789 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:13.101498 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d4d3f-4ecc-46b8-b81d-cd0da23a1789 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:13.103377 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 681/1369] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:06:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:06:13.110340 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462e3bd-ca79-4c9e-85b6-19ac188adfe6 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:13.110867 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462e3bd-ca79-4c9e-85b6-19ac188adfe6 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:13.111280 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462e3bd-ca79-4c9e-85b6-19ac188adfe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:13.148722 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:13.193215 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-a462e3bd-ca79-4c9e-85b6-19ac188adfe6 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:06:13.195084 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-a462e3bd-ca79-4c9e-85b6-19ac188adfe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:13.309593 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 689/1370] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:06:13.646498 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75914e59-50e0-4763-9aca-0554f6193e4b None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:13.646498 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75914e59-50e0-4763-9aca-0554f6193e4b None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:13.646498 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75914e59-50e0-4763-9aca-0554f6193e4b None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:13.649568 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 682/1371] 10.222.0.42 () {54 vars in 964 bytes} [Mon Sep 12 14:06:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:06:13.655591 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e3a61-58ed-46c1-96ec-d67b1e5ef62e None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:13.655978 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e3a61-58ed-46c1-96ec-d67b1e5ef62e None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:13.656356 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e3a61-58ed-46c1-96ec-d67b1e5ef62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:13.737670 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-ae5e3a61-58ed-46c1-96ec-d67b1e5ef62e None None] MFA Rules not processed for user `8da38bc8dd2f4cf798649ce96097a4b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:06:13.739053 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ae5e3a61-58ed-46c1-96ec-d67b1e5ef62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:13.789033 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:13.820903 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 690/1372] 10.222.0.42 () {58 vars in 1061 bytes} [Mon Sep 12 14:06:13 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:06:13.832148 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75914e59-50e0-4763-9aca-0554f6193e4b None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:06:13.833820 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-75914e59-50e0-4763-9aca-0554f6193e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:13.889053 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:13.951071 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75914e59-50e0-4763-9aca-0554f6193e4b None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:06:14.034370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d755b2-5b95-4197-b03d-92506e109d7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:06:14.034972 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d755b2-5b95-4197-b03d-92506e109d7c service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:14.035287 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d755b2-5b95-4197-b03d-92506e109d7c service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:14.035560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d755b2-5b95-4197-b03d-92506e109d7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:14.037382 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-59d755b2-5b95-4197-b03d-92506e109d7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:14.171334 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:14.192509 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d755b2-5b95-4197-b03d-92506e109d7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:06:14.194768 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d755b2-5b95-4197-b03d-92506e109d7c service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:06:14.222441 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 683/1373] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:06:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:06:15.225443 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8333b6fa-f13f-4046-b31a-9ff0ae2f699a None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:15.225951 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8333b6fa-f13f-4046-b31a-9ff0ae2f699a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:15.225951 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8333b6fa-f13f-4046-b31a-9ff0ae2f699a None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:15.227332 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 691/1374] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:06:15.232549 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5d6f9-53ec-464f-922a-d3321c1c058b None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:15.232957 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5d6f9-53ec-464f-922a-d3321c1c058b None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:15.233102 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5d6f9-53ec-464f-922a-d3321c1c058b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:15.299425 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-aed5d6f9-53ec-464f-922a-d3321c1c058b None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:06:15.300657 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-aed5d6f9-53ec-464f-922a-d3321c1c058b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:15.400025 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 684/1375] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:06:15.694670 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8333b6fa-f13f-4046-b31a-9ff0ae2f699a None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:06:15.696621 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-8333b6fa-f13f-4046-b31a-9ff0ae2f699a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:15.860271 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8333b6fa-f13f-4046-b31a-9ff0ae2f699a None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:06:15.913241 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:15.976685 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf195c9-3f67-4438-ad28-c1cd3140fc82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:06:15.977295 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf195c9-3f67-4438-ad28-c1cd3140fc82 service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:15.977570 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf195c9-3f67-4438-ad28-c1cd3140fc82 service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:15.977838 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf195c9-3f67-4438-ad28-c1cd3140fc82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:15.979510 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-5cf195c9-3f67-4438-ad28-c1cd3140fc82 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:16.073200 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf195c9-3f67-4438-ad28-c1cd3140fc82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:06:16.075478 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf195c9-3f67-4438-ad28-c1cd3140fc82 service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:06:16.095394 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 692/1376] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:06:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:06:18.317715 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671935b8-c6a8-45b5-8d16-249d84babc72 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:18.319280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671935b8-c6a8-45b5-8d16-249d84babc72 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:18.319562 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671935b8-c6a8-45b5-8d16-249d84babc72 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:18.321086 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 685/1377] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:06:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:06:18.327204 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10950a-b68e-4e79-800f-b3b9eb7bc3b1 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:18.327643 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10950a-b68e-4e79-800f-b3b9eb7bc3b1 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:18.327737 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10950a-b68e-4e79-800f-b3b9eb7bc3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:18.396619 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-9c10950a-b68e-4e79-800f-b3b9eb7bc3b1 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:06:18.397801 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-9c10950a-b68e-4e79-800f-b3b9eb7bc3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:18.406325 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:18.503641 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 693/1378] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:06:18.793108 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671935b8-c6a8-45b5-8d16-249d84babc72 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:06:18.794604 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-671935b8-c6a8-45b5-8d16-249d84babc72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:18.865504 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:18.953029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671935b8-c6a8-45b5-8d16-249d84babc72 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:06:19.069236 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f7ce1-3567-429d-85af-91aa6ac3f3eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:06:19.069755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f7ce1-3567-429d-85af-91aa6ac3f3eb service nova] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:19.070023 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f7ce1-3567-429d-85af-91aa6ac3f3eb service nova] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:19.070284 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f7ce1-3567-429d-85af-91aa6ac3f3eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:19.071871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-9e2f7ce1-3567-429d-85af-91aa6ac3f3eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:19.092159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:19.166361 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f7ce1-3567-429d-85af-91aa6ac3f3eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:06:19.168173 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f7ce1-3567-429d-85af-91aa6ac3f3eb service nova] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:06:19.187237 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 686/1379] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:06:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:06:21.383993 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e224e-159e-4c4c-89fc-4001ae7b2327 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:21.384588 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e224e-159e-4c4c-89fc-4001ae7b2327 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:21.384588 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e224e-159e-4c4c-89fc-4001ae7b2327 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:21.386716 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 694/1380] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:06:21.391918 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07e203-3348-44ce-bd42-5c9bade9d01b None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:21.392362 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07e203-3348-44ce-bd42-5c9bade9d01b None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:21.392477 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07e203-3348-44ce-bd42-5c9bade9d01b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:21.464835 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-db07e203-3348-44ce-bd42-5c9bade9d01b None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:06:21.466189 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-db07e203-3348-44ce-bd42-5c9bade9d01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:21.580889 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:21.595499 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 687/1381] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:06:21.914767 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e224e-159e-4c4c-89fc-4001ae7b2327 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:06:21.918606 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d92e224e-159e-4c4c-89fc-4001ae7b2327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:22.081037 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:22.091919 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e224e-159e-4c4c-89fc-4001ae7b2327 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:06:22.223757 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2413c20-63de-49f6-9720-cf44caa25f88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:06:22.224525 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2413c20-63de-49f6-9720-cf44caa25f88 service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:22.224913 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2413c20-63de-49f6-9720-cf44caa25f88 service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:22.225308 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2413c20-63de-49f6-9720-cf44caa25f88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:22.227710 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-a2413c20-63de-49f6-9720-cf44caa25f88 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:22.324899 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2413c20-63de-49f6-9720-cf44caa25f88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:06:22.330442 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2413c20-63de-49f6-9720-cf44caa25f88 service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:06:22.342136 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:22.353530 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 695/1382] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:06:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:06:24.646628 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d569e1-349c-4a76-ad4d-c03d55118640 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:24.647162 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d569e1-349c-4a76-ad4d-c03d55118640 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:24.647162 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d569e1-349c-4a76-ad4d-c03d55118640 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:24.648517 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 688/1383] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:06:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:06:24.654437 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29135d-de68-4b34-9020-279e9ad79faa None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:24.654856 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29135d-de68-4b34-9020-279e9ad79faa None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:24.655026 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29135d-de68-4b34-9020-279e9ad79faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:24.720904 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-cf29135d-de68-4b34-9020-279e9ad79faa None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:06:24.722102 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-cf29135d-de68-4b34-9020-279e9ad79faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:24.819468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 696/1384] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:06:25.109867 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d569e1-349c-4a76-ad4d-c03d55118640 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:06:25.111431 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-33d569e1-349c-4a76-ad4d-c03d55118640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:25.264629 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d569e1-349c-4a76-ad4d-c03d55118640 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:06:25.341117 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:25.390828 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322557a5-59a0-4851-8437-c41e70d54548 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:06:25.391536 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322557a5-59a0-4851-8437-c41e70d54548 service nova] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:25.391820 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322557a5-59a0-4851-8437-c41e70d54548 service nova] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:25.392090 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322557a5-59a0-4851-8437-c41e70d54548 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:25.393736 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-322557a5-59a0-4851-8437-c41e70d54548 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:25.492740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322557a5-59a0-4851-8437-c41e70d54548 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:06:25.494337 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322557a5-59a0-4851-8437-c41e70d54548 service nova] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:06:25.512432 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 689/1385] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:06:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:06:27.673218 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5a905-5146-4273-b021-5e6073e12995 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:27.673786 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5a905-5146-4273-b021-5e6073e12995 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:27.673786 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5a905-5146-4273-b021-5e6073e12995 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:27.675487 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 697/1386] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:06:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:06:27.680971 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de833e-87b6-4366-84a7-8197cb8fc0aa None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:27.681473 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de833e-87b6-4366-84a7-8197cb8fc0aa None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:27.681599 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de833e-87b6-4366-84a7-8197cb8fc0aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:27.746308 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-21de833e-87b6-4366-84a7-8197cb8fc0aa None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:06:27.747546 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-21de833e-87b6-4366-84a7-8197cb8fc0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:27.781242 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:27.847981 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 690/1387] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:06:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:06:28.153243 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f5a905-5146-4273-b021-5e6073e12995 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:06:28.154781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-a4f5a905-5146-4273-b021-5e6073e12995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:28.244494 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:28.329628 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f5a905-5146-4273-b021-5e6073e12995 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:06:28.459670 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a2a54-caae-4799-bb0b-b36ccc37d128 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:06:28.460232 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a2a54-caae-4799-bb0b-b36ccc37d128 service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:28.460513 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a2a54-caae-4799-bb0b-b36ccc37d128 service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:28.460785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a2a54-caae-4799-bb0b-b36ccc37d128 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:28.462389 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-611a2a54-caae-4799-bb0b-b36ccc37d128 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:28.501575 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:28.553302 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611a2a54-caae-4799-bb0b-b36ccc37d128 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:06:28.554911 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611a2a54-caae-4799-bb0b-b36ccc37d128 service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:06:28.574601 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 698/1388] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:06:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:06:30.874050 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b29efa-8be8-4e93-a16d-6029783e39f4 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:30.874618 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b29efa-8be8-4e93-a16d-6029783e39f4 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:30.874618 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b29efa-8be8-4e93-a16d-6029783e39f4 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:30.875957 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 691/1389] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:06:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:06:30.881123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4187a3b-0a69-4427-b1ee-4afdd43bfd6f None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:30.881647 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4187a3b-0a69-4427-b1ee-4afdd43bfd6f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:30.881647 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4187a3b-0a69-4427-b1ee-4afdd43bfd6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:30.955684 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-c4187a3b-0a69-4427-b1ee-4afdd43bfd6f None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:06:30.956923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c4187a3b-0a69-4427-b1ee-4afdd43bfd6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:31.055170 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 699/1390] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:06:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:06:31.349710 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b29efa-8be8-4e93-a16d-6029783e39f4 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:06:31.351312 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-39b29efa-8be8-4e93-a16d-6029783e39f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:31.507805 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b29efa-8be8-4e93-a16d-6029783e39f4 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:06:31.525287 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:31.627944 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a34382b-3fa5-43b6-b036-c1e60f4e29ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:06:31.628432 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34382b-3fa5-43b6-b036-c1e60f4e29ad service nova] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:31.628704 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34382b-3fa5-43b6-b036-c1e60f4e29ad service nova] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:31.628972 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34382b-3fa5-43b6-b036-c1e60f4e29ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:31.630577 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-4a34382b-3fa5-43b6-b036-c1e60f4e29ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:31.721350 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a34382b-3fa5-43b6-b036-c1e60f4e29ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:06:31.723026 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a34382b-3fa5-43b6-b036-c1e60f4e29ad service nova] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:06:31.740581 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 692/1391] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:06:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:06:33.898015 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebe1c7-dece-4b48-8100-36dbbbbc78ba None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:33.898517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebe1c7-dece-4b48-8100-36dbbbbc78ba None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:33.898625 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebe1c7-dece-4b48-8100-36dbbbbc78ba None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:33.900129 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 700/1392] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:06:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:06:33.907012 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf07e2-d582-400a-9387-fefd5e5a3749 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:33.907012 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf07e2-d582-400a-9387-fefd5e5a3749 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:33.907614 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf07e2-d582-400a-9387-fefd5e5a3749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:33.932964 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:33.944100 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a8aac-b91b-4145-8664-2871c315eded None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:33.944383 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a8aac-b91b-4145-8664-2871c315eded None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:33.944657 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a8aac-b91b-4145-8664-2871c315eded None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:33.946073 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 701/1393] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:06:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 14:06:33.951003 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1663b-aedc-439d-909c-87fde2b25d52 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:33.951283 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1663b-aedc-439d-909c-87fde2b25d52 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:33.951572 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1663b-aedc-439d-909c-87fde2b25d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:33.966967 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:33.975739 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-51cf07e2-d582-400a-9387-fefd5e5a3749 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:06:33.976972 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-51cf07e2-d582-400a-9387-fefd5e5a3749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:34.017976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-cae1663b-aedc-439d-909c-87fde2b25d52 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:06:34.019243 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-cae1663b-aedc-439d-909c-87fde2b25d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:34.073897 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 693/1394] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:06:34.114366 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 702/1395] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:06:34.374879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a8e22-1daa-48c6-ac18-c3dbc3ecd14f None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:34.375114 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a8e22-1daa-48c6-ac18-c3dbc3ecd14f None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:34.375418 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a8e22-1daa-48c6-ac18-c3dbc3ecd14f None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:34.376773 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 694/1396] 10.222.0.42 () {54 vars in 966 bytes} [Mon Sep 12 14:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:06:34.382102 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f5b31-4fd7-4075-9285-f31538505ba8 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:34.382395 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f5b31-4fd7-4075-9285-f31538505ba8 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:34.382670 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f5b31-4fd7-4075-9285-f31538505ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:34.409914 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852a8e22-1daa-48c6-ac18-c3dbc3ecd14f None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:06:34.411492 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-852a8e22-1daa-48c6-ac18-c3dbc3ecd14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:34.460564 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-a05f5b31-4fd7-4075-9285-f31538505ba8 None None] MFA Rules not processed for user `2092a9639067461c9a837a891f56889b`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:06:34.461793 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-a05f5b31-4fd7-4075-9285-f31538505ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:34.498027 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:34.506817 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:34.544468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 703/1397] 10.222.0.42 () {58 vars in 1063 bytes} [Mon Sep 12 14:06:34 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:06:34.554388 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05f5b31-4fd7-4075-9285-f31538505ba8 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:06:34.561369 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852a8e22-1daa-48c6-ac18-c3dbc3ecd14f None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:06:34.561930 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05f5b31-4fd7-4075-9285-f31538505ba8 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:06:34.636671 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f10bd-0387-4bf5-aca8-745b0e0cc6bf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2092a9639067461c9a837a891f56889b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:06:34.637189 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f10bd-0387-4bf5-aca8-745b0e0cc6bf service glance] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:34.637463 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f10bd-0387-4bf5-aca8-745b0e0cc6bf service glance] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:34.637727 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f10bd-0387-4bf5-aca8-745b0e0cc6bf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:34.639353 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-3a2f10bd-0387-4bf5-aca8-745b0e0cc6bf service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:34.674827 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc23c3d-278f-46f5-ab36-6f25cbda2092 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:06:34.675451 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc23c3d-278f-46f5-ab36-6f25cbda2092 service nova] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:34.675720 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc23c3d-278f-46f5-ab36-6f25cbda2092 service nova] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:34.675984 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc23c3d-278f-46f5-ab36-6f25cbda2092 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:34.677566 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-ecc23c3d-278f-46f5-ab36-6f25cbda2092 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:34.741889 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:34.747019 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:34.751354 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2f10bd-0387-4bf5-aca8-745b0e0cc6bf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:06:34.752502 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2f10bd-0387-4bf5-aca8-745b0e0cc6bf service glance] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:06:34.770528 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc23c3d-278f-46f5-ab36-6f25cbda2092 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:06:34.771223 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 704/1398] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:06:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:06:34.772144 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc23c3d-278f-46f5-ab36-6f25cbda2092 service nova] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:06:34.790229 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 695/1399] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:06:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:06:35.521624 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516f09f-bffa-4faa-8bc9-43620158a1d3 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:35.522552 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516f09f-bffa-4faa-8bc9-43620158a1d3 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:35.522859 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516f09f-bffa-4faa-8bc9-43620158a1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:35.590612 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-1516f09f-bffa-4faa-8bc9-43620158a1d3 None None] MFA Rules not processed for user `1405e758a3af4e8ba659f498cd28ace1`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:06:35.591856 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1516f09f-bffa-4faa-8bc9-43620158a1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:35.679450 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 705/1400] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Sep 12 14:06:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:06:35.727652 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aecb0c-ea6f-48c7-8d29-23f6b8f75112 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:35.728151 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aecb0c-ea6f-48c7-8d29-23f6b8f75112 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:35.728347 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aecb0c-ea6f-48c7-8d29-23f6b8f75112 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:35.729797 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 696/1401] 10.222.0.42 () {54 vars in 943 bytes} [Mon Sep 12 14:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:06:35.736262 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be099ac9-e9a6-47ed-85ab-70f464c18e67 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:35.736553 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be099ac9-e9a6-47ed-85ab-70f464c18e67 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:35.736829 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be099ac9-e9a6-47ed-85ab-70f464c18e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:35.811875 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-be099ac9-e9a6-47ed-85ab-70f464c18e67 None None] MFA Rules not processed for user `25a87ea9c3c5409b97a35c4857ae3ca6`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:06:35.813083 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-be099ac9-e9a6-47ed-85ab-70f464c18e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:35.833396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:35.927753 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 706/1402] 10.222.0.42 () {58 vars in 1040 bytes} [Mon Sep 12 14:06:35 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:06:35.937390 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aecb0c-ea6f-48c7-8d29-23f6b8f75112 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:06:35.939036 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-c3aecb0c-ea6f-48c7-8d29-23f6b8f75112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:36.074185 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aecb0c-ea6f-48c7-8d29-23f6b8f75112 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:06:36.123708 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:36.175377 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95c6a3-57f4-4d73-8455-c8ba32e6faba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a87ea9c3c5409b97a35c4857ae3ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:06:36.175983 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95c6a3-57f4-4d73-8455-c8ba32e6faba service swift] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:36.176259 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95c6a3-57f4-4d73-8455-c8ba32e6faba service swift] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:36.176527 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95c6a3-57f4-4d73-8455-c8ba32e6faba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:36.178152 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-dd95c6a3-57f4-4d73-8455-c8ba32e6faba service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:36.259872 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd95c6a3-57f4-4d73-8455-c8ba32e6faba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:06:36.261989 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd95c6a3-57f4-4d73-8455-c8ba32e6faba service swift] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:06:36.270473 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 697/1403] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Sep 12 14:06:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 14:06:37.018955 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9ad9c-10ef-4c58-a609-09e002e316ad None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:37.019508 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9ad9c-10ef-4c58-a609-09e002e316ad None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:37.019508 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9ad9c-10ef-4c58-a609-09e002e316ad None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:37.020899 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 707/1404] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:06:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:06:37.026781 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803804fa-9925-4150-929d-fc5666428d76 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:37.027263 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803804fa-9925-4150-929d-fc5666428d76 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:37.027375 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803804fa-9925-4150-929d-fc5666428d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:37.103639 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-803804fa-9925-4150-929d-fc5666428d76 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:06:37.105442 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-803804fa-9925-4150-929d-fc5666428d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:37.143854 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:37.216637 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 698/1405] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:06:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:06:37.513320 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9ad9c-10ef-4c58-a609-09e002e316ad None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:06:37.514969 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-48b9ad9c-10ef-4c58-a609-09e002e316ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:37.672371 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:37.689384 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9ad9c-10ef-4c58-a609-09e002e316ad None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:06:37.815685 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf5b27-a465-4835-9791-88c559fa69ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:06:37.816389 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf5b27-a465-4835-9791-88c559fa69ae service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:37.816811 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf5b27-a465-4835-9791-88c559fa69ae service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:37.817074 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf5b27-a465-4835-9791-88c559fa69ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:37.818999 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1edf5b27-a465-4835-9791-88c559fa69ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:37.920587 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf5b27-a465-4835-9791-88c559fa69ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:06:37.922162 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf5b27-a465-4835-9791-88c559fa69ae service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:06:37.935322 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:37.965411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 708/1406] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:06:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:06:40.308412 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df49dbfa-4742-4802-a92e-ab30e4223406 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:40.308976 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df49dbfa-4742-4802-a92e-ab30e4223406 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:40.308976 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df49dbfa-4742-4802-a92e-ab30e4223406 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:40.310429 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 699/1407] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:06:40.319120 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1dbf5c-ac4c-46f1-bfec-4fdd243fdae0 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:40.319642 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1dbf5c-ac4c-46f1-bfec-4fdd243fdae0 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:40.319642 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1dbf5c-ac4c-46f1-bfec-4fdd243fdae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:40.389679 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-cc1dbf5c-ac4c-46f1-bfec-4fdd243fdae0 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:06:40.391243 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-cc1dbf5c-ac4c-46f1-bfec-4fdd243fdae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:40.496553 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 709/1408] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:06:40.788894 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df49dbfa-4742-4802-a92e-ab30e4223406 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:06:40.790503 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-df49dbfa-4742-4802-a92e-ab30e4223406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:40.964008 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df49dbfa-4742-4802-a92e-ab30e4223406 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:06:40.985513 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:41.093128 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475418a-5b99-49ae-abbb-ab570f777f0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:06:41.093646 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475418a-5b99-49ae-abbb-ab570f777f0e service nova] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:41.093923 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475418a-5b99-49ae-abbb-ab570f777f0e service nova] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:41.094189 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475418a-5b99-49ae-abbb-ab570f777f0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:41.095882 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-6475418a-5b99-49ae-abbb-ab570f777f0e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:41.192631 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6475418a-5b99-49ae-abbb-ab570f777f0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:06:41.194667 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6475418a-5b99-49ae-abbb-ab570f777f0e service nova] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:06:41.217675 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 700/1409] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:06:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:06:50.493997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=68 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:51.215236 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=90 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:59.022931 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5396e-51f7-4b39-8b99-744fbc61cbbf None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:59.023520 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5396e-51f7-4b39-8b99-744fbc61cbbf None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:59.023520 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5396e-51f7-4b39-8b99-744fbc61cbbf None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:59.024836 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 710/1410] 10.222.0.42 () {54 vars in 918 bytes} [Mon Sep 12 14:06:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:06:59.030735 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e40b7-41c8-4e80-905b-3b0ad800e69f None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:59.031251 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e40b7-41c8-4e80-905b-3b0ad800e69f None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:59.031358 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e40b7-41c8-4e80-905b-3b0ad800e69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:59.105264 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-b81e40b7-41c8-4e80-905b-3b0ad800e69f None None] MFA Rules not processed for user `ce508bb9aa164a7fb1831a9b5ff5b932`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:06:59.106378 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-b81e40b7-41c8-4e80-905b-3b0ad800e69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:59.209430 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 701/1411] 10.222.0.42 () {58 vars in 1015 bytes} [Mon Sep 12 14:06:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:06:59.224236 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb190056-c529-46fe-8984-810fa0c2383e None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:59.224520 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb190056-c529-46fe-8984-810fa0c2383e None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:59.224795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb190056-c529-46fe-8984-810fa0c2383e None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:59.226121 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 711/1412] 10.222.0.42 () {54 vars in 918 bytes} [Mon Sep 12 14:06:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:06:59.230525 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142981ec-8229-47da-9212-ef4987d888e7 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:06:59.230953 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142981ec-8229-47da-9212-ef4987d888e7 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:06:59.231227 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142981ec-8229-47da-9212-ef4987d888e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:06:59.300642 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:06:59.303842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-142981ec-8229-47da-9212-ef4987d888e7 None None] MFA Rules not processed for user `854ab2dcec5a48b18f409b33cb61b5da`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:06:59.305584 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-142981ec-8229-47da-9212-ef4987d888e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:06:59.419053 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 702/1413] 10.222.0.42 () {58 vars in 1015 bytes} [Mon Sep 12 14:06:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:07:01.998632 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4e962-764c-4650-8bb3-f971ba6ec167 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:01.999192 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4e962-764c-4650-8bb3-f971ba6ec167 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:01.999192 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4e962-764c-4650-8bb3-f971ba6ec167 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:02.000536 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 712/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 14:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:07:02.005739 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9364a-ab8d-42b8-b624-b0af39e9ca6f None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:02.006119 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9364a-ab8d-42b8-b624-b0af39e9ca6f None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:02.006284 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9364a-ab8d-42b8-b624-b0af39e9ca6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:02.076939 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-53c9364a-ab8d-42b8-b624-b0af39e9ca6f None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:07:02.078146 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-53c9364a-ab8d-42b8-b624-b0af39e9ca6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:02.131829 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:02.174105 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 703/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 14:07:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:07:02.467242 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4e962-764c-4650-8bb3-f971ba6ec167 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:07:02.468871 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d9d4e962-764c-4650-8bb3-f971ba6ec167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:02.611076 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4e962-764c-4650-8bb3-f971ba6ec167 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:07:02.686365 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:02.726382 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f4d0ee-356e-41d7-879b-23023328e0d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:07:02.726938 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f4d0ee-356e-41d7-879b-23023328e0d1 service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:02.727215 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f4d0ee-356e-41d7-879b-23023328e0d1 service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:02.727482 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f4d0ee-356e-41d7-879b-23023328e0d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:02.729590 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c5f4d0ee-356e-41d7-879b-23023328e0d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:02.816691 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f4d0ee-356e-41d7-879b-23023328e0d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:07:02.818285 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f4d0ee-356e-41d7-879b-23023328e0d1 service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:07:02.836311 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 713/1416] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:07:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:07:03.964147 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5324c806-bf48-41b0-a247-536f4d13351c None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:03.964652 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5324c806-bf48-41b0-a247-536f4d13351c None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:03.964761 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5324c806-bf48-41b0-a247-536f4d13351c None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:03.966037 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 704/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 14:07:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:07:03.972336 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f5286-f53c-4a69-81a5-0031b8068d94 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:03.972726 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f5286-f53c-4a69-81a5-0031b8068d94 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:03.972898 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f5286-f53c-4a69-81a5-0031b8068d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:04.049214 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-c24f5286-f53c-4a69-81a5-0031b8068d94 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:07:04.050412 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c24f5286-f53c-4a69-81a5-0031b8068d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:04.077910 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:04.147785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 714/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 14:07:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:07:04.294853 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5324c806-bf48-41b0-a247-536f4d13351c None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:07:04.296361 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-5324c806-bf48-41b0-a247-536f4d13351c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:04.440639 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5324c806-bf48-41b0-a247-536f4d13351c None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:07:04.480231 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:04.554579 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0886c59-3105-4086-ae76-ceb48dc51ab4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:07:04.555210 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0886c59-3105-4086-ae76-ceb48dc51ab4 service nova] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:04.555490 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0886c59-3105-4086-ae76-ceb48dc51ab4 service nova] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:04.555760 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0886c59-3105-4086-ae76-ceb48dc51ab4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:04.557396 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-c0886c59-3105-4086-ae76-ceb48dc51ab4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:04.646005 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0886c59-3105-4086-ae76-ceb48dc51ab4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:07:04.647616 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0886c59-3105-4086-ae76-ceb48dc51ab4 service nova] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:07:04.666520 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 705/1419] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:07:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:07:05.790045 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b709e-9a41-4e7e-bae5-699040d131d6 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:05.791205 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b709e-9a41-4e7e-bae5-699040d131d6 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:05.791205 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b709e-9a41-4e7e-bae5-699040d131d6 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:05.792468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 715/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 14:07:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:07:05.798569 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7523f3-c382-4664-b2ba-58e33ad69954 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:05.798977 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7523f3-c382-4664-b2ba-58e33ad69954 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:05.799152 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7523f3-c382-4664-b2ba-58e33ad69954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:05.853336 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:05.875491 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-1c7523f3-c382-4664-b2ba-58e33ad69954 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:07:05.876693 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-1c7523f3-c382-4664-b2ba-58e33ad69954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:05.970913 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 706/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 14:07:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:07:06.123262 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b709e-9a41-4e7e-bae5-699040d131d6 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:07:06.124842 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-318b709e-9a41-4e7e-bae5-699040d131d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:06.380580 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b709e-9a41-4e7e-bae5-699040d131d6 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:07:06.393332 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:06.489534 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af1ba6-bd07-4cd0-adcb-f8cbe5e63d05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:07:06.490131 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af1ba6-bd07-4cd0-adcb-f8cbe5e63d05 service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:06.490432 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af1ba6-bd07-4cd0-adcb-f8cbe5e63d05 service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:06.490696 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af1ba6-bd07-4cd0-adcb-f8cbe5e63d05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:06.492292 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f5af1ba6-bd07-4cd0-adcb-f8cbe5e63d05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:06.581711 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af1ba6-bd07-4cd0-adcb-f8cbe5e63d05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:07:06.583319 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af1ba6-bd07-4cd0-adcb-f8cbe5e63d05 service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:07:06.600845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 716/1422] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:07:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:07:07.722962 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53f26f-786e-4822-b1fb-b0be97ae842a None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:07.723528 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53f26f-786e-4822-b1fb-b0be97ae842a None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:07.723528 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53f26f-786e-4822-b1fb-b0be97ae842a None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:07.724847 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 707/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 14:07:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:07:07.731499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb606f-4d5f-4663-ad67-d7990a27fdc1 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:07.731888 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb606f-4d5f-4663-ad67-d7990a27fdc1 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:07.732056 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb606f-4d5f-4663-ad67-d7990a27fdc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:07.752126 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:07.807176 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-41fb606f-4d5f-4663-ad67-d7990a27fdc1 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:07:07.808387 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-41fb606f-4d5f-4663-ad67-d7990a27fdc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:07.905749 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 717/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 14:07:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:07:08.053036 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe53f26f-786e-4822-b1fb-b0be97ae842a None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:07:08.054546 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-fe53f26f-786e-4822-b1fb-b0be97ae842a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:08.166700 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:08.202409 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe53f26f-786e-4822-b1fb-b0be97ae842a None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:07:08.317644 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed1904-4bce-42a6-aafb-6ac287f0676e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:07:08.318184 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed1904-4bce-42a6-aafb-6ac287f0676e service nova] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:08.318718 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed1904-4bce-42a6-aafb-6ac287f0676e service nova] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:08.319024 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed1904-4bce-42a6-aafb-6ac287f0676e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:08.320629 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-3aed1904-4bce-42a6-aafb-6ac287f0676e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:08.396486 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:08.410533 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed1904-4bce-42a6-aafb-6ac287f0676e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:07:08.412135 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed1904-4bce-42a6-aafb-6ac287f0676e service nova] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:07:08.429403 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 708/1425] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:07:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:07:09.529176 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee1051-9a60-4892-9e5f-2bf887ed66e1 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:09.529743 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee1051-9a60-4892-9e5f-2bf887ed66e1 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:09.529743 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee1051-9a60-4892-9e5f-2bf887ed66e1 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:09.531099 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 718/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 14:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:07:09.537188 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351a548-6eed-44b5-8f21-193e83d1460b None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:09.537660 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351a548-6eed-44b5-8f21-193e83d1460b None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:09.537769 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351a548-6eed-44b5-8f21-193e83d1460b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:09.612493 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-8351a548-6eed-44b5-8f21-193e83d1460b None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:07:09.613700 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-8351a548-6eed-44b5-8f21-193e83d1460b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:09.708910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 709/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 14:07:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:07:09.858462 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee1051-9a60-4892-9e5f-2bf887ed66e1 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:07:09.860006 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1aee1051-9a60-4892-9e5f-2bf887ed66e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:09.943092 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:10.003461 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee1051-9a60-4892-9e5f-2bf887ed66e1 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:07:10.117389 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d054a-a8d2-4492-b08e-6d6bb1c0dc4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:07:10.117917 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d054a-a8d2-4492-b08e-6d6bb1c0dc4c service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:10.118196 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d054a-a8d2-4492-b08e-6d6bb1c0dc4c service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:10.119030 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d054a-a8d2-4492-b08e-6d6bb1c0dc4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:10.120618 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d08d054a-a8d2-4492-b08e-6d6bb1c0dc4c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:10.168077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:10.210007 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08d054a-a8d2-4492-b08e-6d6bb1c0dc4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:07:10.211610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08d054a-a8d2-4492-b08e-6d6bb1c0dc4c service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:07:10.230339 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 719/1428] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:07:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:07:11.285208 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1fa1cd-c23c-4d63-bf02-d4c934731594 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:11.285733 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1fa1cd-c23c-4d63-bf02-d4c934731594 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:11.285733 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1fa1cd-c23c-4d63-bf02-d4c934731594 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:11.287118 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 710/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 14:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:07:11.293363 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb86fb6-d596-450c-8380-576853a1b147 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:11.293808 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb86fb6-d596-450c-8380-576853a1b147 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:11.293923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb86fb6-d596-450c-8380-576853a1b147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:11.368136 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-9cb86fb6-d596-450c-8380-576853a1b147 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:07:11.369907 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-9cb86fb6-d596-450c-8380-576853a1b147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:11.462213 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 720/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 14:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:07:11.611039 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1fa1cd-c23c-4d63-bf02-d4c934731594 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:07:11.612482 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-4a1fa1cd-c23c-4d63-bf02-d4c934731594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:11.660472 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:11.756254 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1fa1cd-c23c-4d63-bf02-d4c934731594 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:07:11.865989 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f5c2a6-0c98-4ab0-8b57-100c3b9060bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:07:11.866533 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5c2a6-0c98-4ab0-8b57-100c3b9060bd service nova] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:11.866804 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5c2a6-0c98-4ab0-8b57-100c3b9060bd service nova] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:11.867097 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5c2a6-0c98-4ab0-8b57-100c3b9060bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:11.869294 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-a6f5c2a6-0c98-4ab0-8b57-100c3b9060bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:11.876008 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:11.959149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f5c2a6-0c98-4ab0-8b57-100c3b9060bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:07:11.960740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f5c2a6-0c98-4ab0-8b57-100c3b9060bd service nova] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:07:11.978594 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 711/1431] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:07:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:07:13.021202 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f201e7-2b3a-4511-a1d3-fabfac800a9f None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:13.021852 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f201e7-2b3a-4511-a1d3-fabfac800a9f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:13.021852 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f201e7-2b3a-4511-a1d3-fabfac800a9f None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:13.023106 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 721/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 14:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:07:13.029215 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b6723-be2b-40f2-80dd-5b22fe1aaa92 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:13.029608 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b6723-be2b-40f2-80dd-5b22fe1aaa92 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:13.029767 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b6723-be2b-40f2-80dd-5b22fe1aaa92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:13.104899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-378b6723-be2b-40f2-80dd-5b22fe1aaa92 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:07:13.106025 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-378b6723-be2b-40f2-80dd-5b22fe1aaa92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:13.173393 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:13.202231 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 712/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 14:07:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:07:13.351073 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f201e7-2b3a-4511-a1d3-fabfac800a9f None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:07:13.352597 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f7f201e7-2b3a-4511-a1d3-fabfac800a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:13.369704 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:13.494271 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f201e7-2b3a-4511-a1d3-fabfac800a9f None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:07:13.579649 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:13.604562 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34dae8a-ca57-42a4-a72c-5b1a2a246ef4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:07:13.605165 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34dae8a-ca57-42a4-a72c-5b1a2a246ef4 service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:13.605440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34dae8a-ca57-42a4-a72c-5b1a2a246ef4 service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:13.605705 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34dae8a-ca57-42a4-a72c-5b1a2a246ef4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:13.607316 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b34dae8a-ca57-42a4-a72c-5b1a2a246ef4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:13.695335 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34dae8a-ca57-42a4-a72c-5b1a2a246ef4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:07:13.697399 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34dae8a-ca57-42a4-a72c-5b1a2a246ef4 service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:07:13.714794 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 722/1434] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:07:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:07:14.771747 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d797ada-66ae-403e-98bb-7a4f133f5691 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:14.772319 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d797ada-66ae-403e-98bb-7a4f133f5691 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:14.772319 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d797ada-66ae-403e-98bb-7a4f133f5691 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:14.773627 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 713/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 14:07:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:07:14.779683 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e741324e-b0e3-4f59-a9d5-2571afc3cc31 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:14.780121 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e741324e-b0e3-4f59-a9d5-2571afc3cc31 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:14.780267 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e741324e-b0e3-4f59-a9d5-2571afc3cc31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:14.856478 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-e741324e-b0e3-4f59-a9d5-2571afc3cc31 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:07:14.857679 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e741324e-b0e3-4f59-a9d5-2571afc3cc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:14.897920 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:14.951985 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 723/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 14:07:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:07:15.100452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d797ada-66ae-403e-98bb-7a4f133f5691 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:07:15.101885 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-7d797ada-66ae-403e-98bb-7a4f133f5691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:15.244317 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d797ada-66ae-403e-98bb-7a4f133f5691 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:07:15.297686 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:15.355445 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7367ccb-660f-4d81-85ac-21de8e290b00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:07:15.355973 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7367ccb-660f-4d81-85ac-21de8e290b00 service nova] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:15.356246 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7367ccb-660f-4d81-85ac-21de8e290b00 service nova] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:15.356512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7367ccb-660f-4d81-85ac-21de8e290b00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:15.358085 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-b7367ccb-660f-4d81-85ac-21de8e290b00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:15.449038 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7367ccb-660f-4d81-85ac-21de8e290b00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:07:15.450626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7367ccb-660f-4d81-85ac-21de8e290b00 service nova] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:07:15.468736 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 714/1437] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:07:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:07:16.512510 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bfbfd-0ffb-4e66-a845-2ab64e2a584c None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:16.513112 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bfbfd-0ffb-4e66-a845-2ab64e2a584c None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:16.513112 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bfbfd-0ffb-4e66-a845-2ab64e2a584c None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:16.514396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 724/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 14:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 14:07:16.520678 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad10ca5-6e5a-449c-addc-af4431f06a4e None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:16.521131 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad10ca5-6e5a-449c-addc-af4431f06a4e None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:16.521338 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad10ca5-6e5a-449c-addc-af4431f06a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:16.597509 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-cad10ca5-6e5a-449c-addc-af4431f06a4e None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:07:16.598707 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-cad10ca5-6e5a-449c-addc-af4431f06a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:16.600732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:16.701405 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 715/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 14:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:07:16.850568 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492bfbfd-0ffb-4e66-a845-2ab64e2a584c None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:07:16.852091 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-492bfbfd-0ffb-4e66-a845-2ab64e2a584c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:16.994520 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492bfbfd-0ffb-4e66-a845-2ab64e2a584c None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:07:17.020884 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:17.106554 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08b5b7-1d64-4dda-8adb-a91117a30d33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:07:17.107165 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08b5b7-1d64-4dda-8adb-a91117a30d33 service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:17.107441 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08b5b7-1d64-4dda-8adb-a91117a30d33 service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:17.107706 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08b5b7-1d64-4dda-8adb-a91117a30d33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:17.109284 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1e08b5b7-1d64-4dda-8adb-a91117a30d33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:17.195707 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e08b5b7-1d64-4dda-8adb-a91117a30d33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:07:17.197835 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e08b5b7-1d64-4dda-8adb-a91117a30d33 service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:07:17.215553 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 725/1440] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:07:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:07:19.303644 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a2730-b908-416c-a3f9-657b0b0b5808 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:19.304220 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a2730-b908-416c-a3f9-657b0b0b5808 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:19.304220 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a2730-b908-416c-a3f9-657b0b0b5808 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:19.305583 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 716/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 14:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:07:19.310808 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7faab9-4740-4ec7-8dd2-d9f0fd21e43f None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:19.311317 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7faab9-4740-4ec7-8dd2-d9f0fd21e43f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:19.311420 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7faab9-4740-4ec7-8dd2-d9f0fd21e43f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:19.355414 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:19.380244 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-fc7faab9-4740-4ec7-8dd2-d9f0fd21e43f None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:07:19.381462 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-fc7faab9-4740-4ec7-8dd2-d9f0fd21e43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:19.476022 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 726/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 14:07:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:07:19.759730 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a2730-b908-416c-a3f9-657b0b0b5808 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:07:19.761271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-c41a2730-b908-416c-a3f9-657b0b0b5808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:19.862828 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a2730-b908-416c-a3f9-657b0b0b5808 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:07:19.894762 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:19.937390 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2047f184-26bc-4d51-b68e-369e2aa9a012 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:07:19.937996 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047f184-26bc-4d51-b68e-369e2aa9a012 service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:19.938278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047f184-26bc-4d51-b68e-369e2aa9a012 service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:19.938545 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047f184-26bc-4d51-b68e-369e2aa9a012 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:19.940166 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-2047f184-26bc-4d51-b68e-369e2aa9a012 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:20.046464 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2047f184-26bc-4d51-b68e-369e2aa9a012 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:07:20.047993 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2047f184-26bc-4d51-b68e-369e2aa9a012 service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:07:20.067395 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 717/1443] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:07:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:07:22.427260 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d910b2b-c47b-4275-bc37-7b9893df8d31 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:22.427834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d910b2b-c47b-4275-bc37-7b9893df8d31 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:22.427834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d910b2b-c47b-4275-bc37-7b9893df8d31 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:22.429223 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 727/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 14:07:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:07:22.435148 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a14b7-b93e-4cd4-bf72-1f6409802040 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:22.435553 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a14b7-b93e-4cd4-bf72-1f6409802040 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:22.435759 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a14b7-b93e-4cd4-bf72-1f6409802040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:22.505337 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-b58a14b7-b93e-4cd4-bf72-1f6409802040 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:07:22.506533 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-b58a14b7-b93e-4cd4-bf72-1f6409802040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:22.519053 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:22.604868 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 718/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 14:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:07:22.893838 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d910b2b-c47b-4275-bc37-7b9893df8d31 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:07:22.895469 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-2d910b2b-c47b-4275-bc37-7b9893df8d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:22.997545 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d910b2b-c47b-4275-bc37-7b9893df8d31 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:07:23.010544 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:23.069910 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47011b89-346c-43a4-9bcc-4bc7d521217b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:07:23.070512 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47011b89-346c-43a4-9bcc-4bc7d521217b service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:23.070793 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47011b89-346c-43a4-9bcc-4bc7d521217b service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:23.071607 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47011b89-346c-43a4-9bcc-4bc7d521217b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:23.073285 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-47011b89-346c-43a4-9bcc-4bc7d521217b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:23.179324 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47011b89-346c-43a4-9bcc-4bc7d521217b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:07:23.180677 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47011b89-346c-43a4-9bcc-4bc7d521217b service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:07:23.199386 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 728/1446] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:07:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:07:25.640575 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a1732-8d02-4570-b99c-59fffb605102 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:25.641137 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a1732-8d02-4570-b99c-59fffb605102 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:25.641137 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a1732-8d02-4570-b99c-59fffb605102 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:25.642460 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 719/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 14:07:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:07:25.647777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf9cd4a-55cf-4890-91db-048bfc2c789c None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:25.648176 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf9cd4a-55cf-4890-91db-048bfc2c789c None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:25.648337 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf9cd4a-55cf-4890-91db-048bfc2c789c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:25.696408 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:25.722034 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-dbf9cd4a-55cf-4890-91db-048bfc2c789c None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:07:25.723719 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-dbf9cd4a-55cf-4890-91db-048bfc2c789c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:25.817372 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 729/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 14:07:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:07:26.110966 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169a1732-8d02-4570-b99c-59fffb605102 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:07:26.112497 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-169a1732-8d02-4570-b99c-59fffb605102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:26.212935 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169a1732-8d02-4570-b99c-59fffb605102 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:07:26.252557 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:26.285933 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167a1f8-0053-4be7-82ed-35a0eb079fac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:07:26.286527 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167a1f8-0053-4be7-82ed-35a0eb079fac service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:26.286797 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167a1f8-0053-4be7-82ed-35a0eb079fac service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:26.287088 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167a1f8-0053-4be7-82ed-35a0eb079fac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:26.288665 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-d167a1f8-0053-4be7-82ed-35a0eb079fac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:26.399765 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d167a1f8-0053-4be7-82ed-35a0eb079fac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:07:26.401193 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d167a1f8-0053-4be7-82ed-35a0eb079fac service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:07:26.420539 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 720/1449] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:07:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:07:28.684688 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc97ac5-6192-4d6b-8de5-920b456b18dc None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:28.685273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc97ac5-6192-4d6b-8de5-920b456b18dc None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:28.685273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc97ac5-6192-4d6b-8de5-920b456b18dc None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:28.686579 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 730/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 14:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:07:28.692190 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76a8e2-a1dc-4f54-a2f3-b28a5a15fefc None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:28.692625 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76a8e2-a1dc-4f54-a2f3-b28a5a15fefc None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:28.692739 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76a8e2-a1dc-4f54-a2f3-b28a5a15fefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:28.757319 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-4f76a8e2-a1dc-4f54-a2f3-b28a5a15fefc None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:07:28.758550 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-4f76a8e2-a1dc-4f54-a2f3-b28a5a15fefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:28.778824 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:28.856115 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 721/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 14:07:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:07:29.139535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc97ac5-6192-4d6b-8de5-920b456b18dc None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:07:29.141069 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-fbc97ac5-6192-4d6b-8de5-920b456b18dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:29.242396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc97ac5-6192-4d6b-8de5-920b456b18dc None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:07:29.262722 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:29.314952 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6afa88b-4b8c-43f1-a8f6-58dc8abd039a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:07:29.315549 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6afa88b-4b8c-43f1-a8f6-58dc8abd039a service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:29.315826 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6afa88b-4b8c-43f1-a8f6-58dc8abd039a service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:29.316090 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6afa88b-4b8c-43f1-a8f6-58dc8abd039a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:29.317700 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f6afa88b-4b8c-43f1-a8f6-58dc8abd039a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:29.429567 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6afa88b-4b8c-43f1-a8f6-58dc8abd039a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:07:29.431491 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6afa88b-4b8c-43f1-a8f6-58dc8abd039a service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:07:29.449647 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 731/1452] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:07:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:07:31.655632 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e774a6c-c7b4-4dba-a808-ace3d42de6aa None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:31.656170 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e774a6c-c7b4-4dba-a808-ace3d42de6aa None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:31.656170 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e774a6c-c7b4-4dba-a808-ace3d42de6aa None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:31.657514 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 722/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 14:07:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:07:31.663076 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e7899-36e2-4854-abdd-c8c4a28bbdbf None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:31.663521 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e7899-36e2-4854-abdd-c8c4a28bbdbf None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:31.663640 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e7899-36e2-4854-abdd-c8c4a28bbdbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:31.727814 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:31.730399 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-e03e7899-36e2-4854-abdd-c8c4a28bbdbf None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:07:31.732191 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e03e7899-36e2-4854-abdd-c8c4a28bbdbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:31.834744 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 732/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 14:07:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:07:32.131045 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e774a6c-c7b4-4dba-a808-ace3d42de6aa None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:07:32.132628 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-1e774a6c-c7b4-4dba-a808-ace3d42de6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:32.236418 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e774a6c-c7b4-4dba-a808-ace3d42de6aa None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:07:32.284016 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:32.308404 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f27a4f-5e7c-4039-a727-332c09980f25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:07:32.309002 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f27a4f-5e7c-4039-a727-332c09980f25 service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:32.309275 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f27a4f-5e7c-4039-a727-332c09980f25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:32.309543 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f27a4f-5e7c-4039-a727-332c09980f25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:32.311138 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-f1f27a4f-5e7c-4039-a727-332c09980f25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:32.417922 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f27a4f-5e7c-4039-a727-332c09980f25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:07:32.419402 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f27a4f-5e7c-4039-a727-332c09980f25 service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:07:32.437340 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 723/1455] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:07:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:07:34.639519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5aa19-ae22-449c-bb22-978b998791b3 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:34.640093 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5aa19-ae22-449c-bb22-978b998791b3 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:34.640093 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5aa19-ae22-449c-bb22-978b998791b3 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:34.641414 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 733/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 14:07:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:07:34.646770 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44031ab6-6a41-4271-8bdf-90245c5c2b84 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:34.647189 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44031ab6-6a41-4271-8bdf-90245c5c2b84 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:34.647359 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44031ab6-6a41-4271-8bdf-90245c5c2b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:34.713251 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-44031ab6-6a41-4271-8bdf-90245c5c2b84 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:07:34.714446 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-44031ab6-6a41-4271-8bdf-90245c5c2b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:34.744044 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:34.811277 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 724/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 14:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:07:35.095899 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5aa19-ae22-449c-bb22-978b998791b3 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:07:35.097459 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b4e5aa19-ae22-449c-bb22-978b998791b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:35.198512 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5aa19-ae22-449c-bb22-978b998791b3 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:07:35.226868 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:35.273054 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf95b56-9c6d-44c6-bbe0-978d5545e2f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:07:35.274298 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf95b56-9c6d-44c6-bbe0-978d5545e2f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:35.274579 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf95b56-9c6d-44c6-bbe0-978d5545e2f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:35.274860 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf95b56-9c6d-44c6-bbe0-978d5545e2f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:35.276496 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1cf95b56-9c6d-44c6-bbe0-978d5545e2f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:35.383605 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf95b56-9c6d-44c6-bbe0-978d5545e2f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:07:35.384948 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf95b56-9c6d-44c6-bbe0-978d5545e2f5 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:07:35.402428 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 734/1458] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:07:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:07:37.623471 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb9046-e46c-450c-b855-aea496f343b9 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:37.624045 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb9046-e46c-450c-b855-aea496f343b9 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:37.624045 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb9046-e46c-450c-b855-aea496f343b9 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:37.625362 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 725/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 14:07:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:07:37.630671 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c060f4e-7b4d-4392-9b7e-7979432fb7a5 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:37.631123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c060f4e-7b4d-4392-9b7e-7979432fb7a5 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:37.631266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c060f4e-7b4d-4392-9b7e-7979432fb7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:37.704222 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-0c060f4e-7b4d-4392-9b7e-7979432fb7a5 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:07:37.705452 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-0c060f4e-7b4d-4392-9b7e-7979432fb7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:37.714360 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:37.801696 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 735/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 14:07:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:07:38.086916 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb9046-e46c-450c-b855-aea496f343b9 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:07:38.089402 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-edeb9046-e46c-450c-b855-aea496f343b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:38.195801 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb9046-e46c-450c-b855-aea496f343b9 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:07:38.252073 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:38.272145 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1608a6-dddf-41fb-8e0a-c95adb308994 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:07:38.272773 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1608a6-dddf-41fb-8e0a-c95adb308994 service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:38.273049 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1608a6-dddf-41fb-8e0a-c95adb308994 service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:38.273319 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1608a6-dddf-41fb-8e0a-c95adb308994 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:38.275550 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-cb1608a6-dddf-41fb-8e0a-c95adb308994 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:38.383777 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1608a6-dddf-41fb-8e0a-c95adb308994 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:07:38.385204 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1608a6-dddf-41fb-8e0a-c95adb308994 service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:07:38.403119 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 726/1461] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:07:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:07:40.601475 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f9eab-ee68-4e0e-9dac-e6f43fb3142f None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:40.602045 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f9eab-ee68-4e0e-9dac-e6f43fb3142f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:40.602045 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f9eab-ee68-4e0e-9dac-e6f43fb3142f None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:40.603417 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 736/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 14:07:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:07:40.608829 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df312adf-3ad5-4be1-8a24-7d2440452d07 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:40.609245 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df312adf-3ad5-4be1-8a24-7d2440452d07 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:40.609380 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df312adf-3ad5-4be1-8a24-7d2440452d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:40.676330 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-df312adf-3ad5-4be1-8a24-7d2440452d07 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:07:40.678233 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-df312adf-3ad5-4be1-8a24-7d2440452d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:40.719082 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:40.776901 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 727/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 14:07:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:07:41.063030 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f9eab-ee68-4e0e-9dac-e6f43fb3142f None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:07:41.064591 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-237f9eab-ee68-4e0e-9dac-e6f43fb3142f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:41.166560 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f9eab-ee68-4e0e-9dac-e6f43fb3142f None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:07:41.204309 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:41.239483 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f819bc-6b2b-4276-8be6-413d810d6d34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:07:41.240086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f819bc-6b2b-4276-8be6-413d810d6d34 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:41.240366 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f819bc-6b2b-4276-8be6-413d810d6d34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:41.240640 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f819bc-6b2b-4276-8be6-413d810d6d34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:41.242874 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d2f819bc-6b2b-4276-8be6-413d810d6d34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:41.350411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f819bc-6b2b-4276-8be6-413d810d6d34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:07:41.352392 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f819bc-6b2b-4276-8be6-413d810d6d34 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:07:41.369888 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 737/1464] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:07:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:07:43.491846 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c2734-b3db-4104-a7c0-fd4a6ccad0cc None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:43.492411 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c2734-b3db-4104-a7c0-fd4a6ccad0cc None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:43.492411 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c2734-b3db-4104-a7c0-fd4a6ccad0cc None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:43.493756 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 728/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 14:07:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:07:43.499033 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d25b9-d4d5-4389-b00f-2183609576ac None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:43.499429 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d25b9-d4d5-4389-b00f-2183609576ac None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:43.499593 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d25b9-d4d5-4389-b00f-2183609576ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:43.565308 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-068d25b9-d4d5-4389-b00f-2183609576ac None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:07:43.566513 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-068d25b9-d4d5-4389-b00f-2183609576ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:43.585189 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:43.662989 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 738/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 14:07:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:07:43.945331 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c2734-b3db-4104-a7c0-fd4a6ccad0cc None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:07:43.946908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-192c2734-b3db-4104-a7c0-fd4a6ccad0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:44.047479 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c2734-b3db-4104-a7c0-fd4a6ccad0cc None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:07:44.113941 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:44.121538 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ae5c2-c56b-4318-9a64-ff6ea21d728c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:07:44.122139 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ae5c2-c56b-4318-9a64-ff6ea21d728c service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:44.122411 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ae5c2-c56b-4318-9a64-ff6ea21d728c service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:44.122675 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ae5c2-c56b-4318-9a64-ff6ea21d728c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:44.124280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-315ae5c2-c56b-4318-9a64-ff6ea21d728c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:44.233308 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ae5c2-c56b-4318-9a64-ff6ea21d728c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:07:44.234734 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ae5c2-c56b-4318-9a64-ff6ea21d728c service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:07:44.253071 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 729/1467] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:07:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:07:46.435166 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1eb891-4e8a-4832-af04-e0fb81b14b76 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:46.435712 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1eb891-4e8a-4832-af04-e0fb81b14b76 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:46.435712 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1eb891-4e8a-4832-af04-e0fb81b14b76 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:46.437068 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 739/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 14:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:07:46.442294 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e85322-cc5c-4853-ae37-221fe7f0ac1a None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:46.442711 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e85322-cc5c-4853-ae37-221fe7f0ac1a None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:46.442861 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e85322-cc5c-4853-ae37-221fe7f0ac1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:46.508051 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-b2e85322-cc5c-4853-ae37-221fe7f0ac1a None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:07:46.509251 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-b2e85322-cc5c-4853-ae37-221fe7f0ac1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:46.557305 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:46.606982 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 730/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 14:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:07:46.890452 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1eb891-4e8a-4832-af04-e0fb81b14b76 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:07:46.892023 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-9f1eb891-4e8a-4832-af04-e0fb81b14b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:46.992206 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1eb891-4e8a-4832-af04-e0fb81b14b76 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:07:47.038388 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:47.063346 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a588e-031b-4f06-83de-2d513e910057 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:07:47.063945 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a588e-031b-4f06-83de-2d513e910057 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:07:47.064220 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a588e-031b-4f06-83de-2d513e910057 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:07:47.064490 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a588e-031b-4f06-83de-2d513e910057 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:07:47.066064 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-fc2a588e-031b-4f06-83de-2d513e910057 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:07:47.172010 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2a588e-031b-4f06-83de-2d513e910057 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:07:47.173414 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2a588e-031b-4f06-83de-2d513e910057 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:07:47.190607 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 740/1470] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:07:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:07:56.605468 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=22 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:07:57.188400 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=66 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:08:07.096409 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea51f8-cd66-4a7e-a2ea-f16dde618265 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:08:07.096952 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea51f8-cd66-4a7e-a2ea-f16dde618265 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:08:07.096952 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea51f8-cd66-4a7e-a2ea-f16dde618265 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:08:07.098230 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 731/1471] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:08:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:08:07.103608 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cffecf-0ac2-47b5-8f54-b7e29986fb66 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:08:07.104009 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cffecf-0ac2-47b5-8f54-b7e29986fb66 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:08:07.104191 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cffecf-0ac2-47b5-8f54-b7e29986fb66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:08:07.169049 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-58cffecf-0ac2-47b5-8f54-b7e29986fb66 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:08:07.170253 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-58cffecf-0ac2-47b5-8f54-b7e29986fb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:08:07.265790 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 741/1472] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:08:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:08:07.573806 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b62a9-82a7-400f-a345-690536815c50 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:08:07.574093 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b62a9-82a7-400f-a345-690536815c50 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:08:07.574367 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b62a9-82a7-400f-a345-690536815c50 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:08:07.575732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 732/1473] 10.222.0.42 () {54 vars in 966 bytes} [Mon Sep 12 14:08:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:08:07.580759 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44410be-9152-426f-91a6-c4e4d3cf4374 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:08:07.581041 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44410be-9152-426f-91a6-c4e4d3cf4374 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:08:07.581313 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44410be-9152-426f-91a6-c4e4d3cf4374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:08:07.657503 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-f44410be-9152-426f-91a6-c4e4d3cf4374 None None] MFA Rules not processed for user `2092a9639067461c9a837a891f56889b`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:08:07.658722 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f44410be-9152-426f-91a6-c4e4d3cf4374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:08:07.679594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:08:07.735459 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 742/1474] 10.222.0.42 () {58 vars in 1063 bytes} [Mon Sep 12 14:08:07 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:08:07.743190 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b62a9-82a7-400f-a345-690536815c50 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:08:07.744744 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-786b62a9-82a7-400f-a345-690536815c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:08:07.845665 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b62a9-82a7-400f-a345-690536815c50 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:08:07.919112 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53128110-c1b8-42b1-bf43-6bdb34b17ffa service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2092a9639067461c9a837a891f56889b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:08:07.919714 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53128110-c1b8-42b1-bf43-6bdb34b17ffa service glance] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:08:07.919987 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53128110-c1b8-42b1-bf43-6bdb34b17ffa service glance] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:08:07.920248 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53128110-c1b8-42b1-bf43-6bdb34b17ffa service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:08:07.921855 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-53128110-c1b8-42b1-bf43-6bdb34b17ffa service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:08:08.093341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:08:08.148967 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53128110-c1b8-42b1-bf43-6bdb34b17ffa service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:08:08.150382 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53128110-c1b8-42b1-bf43-6bdb34b17ffa service glance] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:08:08.168684 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 733/1475] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:08:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:08:10.386940 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9dff4-fade-4d86-8ff7-e7de393af09d None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:08:10.387513 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9dff4-fade-4d86-8ff7-e7de393af09d None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:08:10.387513 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9dff4-fade-4d86-8ff7-e7de393af09d None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:08:10.388850 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 743/1476] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:08:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:08:10.394013 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cc5d7-c3ae-4d8e-9da5-9aaf584f9696 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:08:10.394460 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cc5d7-c3ae-4d8e-9da5-9aaf584f9696 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:08:10.394558 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cc5d7-c3ae-4d8e-9da5-9aaf584f9696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:08:10.459621 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-1b5cc5d7-c3ae-4d8e-9da5-9aaf584f9696 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:08:10.460835 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-1b5cc5d7-c3ae-4d8e-9da5-9aaf584f9696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:08:10.555005 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 734/1477] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:08:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:08:10.824876 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f9dff4-fade-4d86-8ff7-e7de393af09d None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:08:10.826463 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-98f9dff4-fade-4d86-8ff7-e7de393af09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:08:10.948698 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f9dff4-fade-4d86-8ff7-e7de393af09d None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:08:10.996428 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:08:11.040012 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbcef93-7c4c-4e45-8121-ea952ec35e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:08:11.040594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbcef93-7c4c-4e45-8121-ea952ec35e73 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:08:11.040875 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbcef93-7c4c-4e45-8121-ea952ec35e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:08:11.041142 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbcef93-7c4c-4e45-8121-ea952ec35e73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:08:11.041781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbcef93-7c4c-4e45-8121-ea952ec35e73 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:08:11.043522 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: WARNING py.warnings [None req-9cbcef93-7c4c-4e45-8121-ea952ec35e73 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 14:08:11.043522 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: warnings.warn(msg) Sep 12 14:08:11.043522 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: Sep 12 14:08:11.044647 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbcef93-7c4c-4e45-8121-ea952ec35e73 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:08:11.063147 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 744/1478] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 12 14:08:10 2022] GET /identity/v3/projects => generated 2553 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:08:13.218431 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537a0a6-d827-4535-98b2-44d508b98bc8 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:08:13.218986 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537a0a6-d827-4535-98b2-44d508b98bc8 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:08:13.218986 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537a0a6-d827-4535-98b2-44d508b98bc8 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:08:13.220296 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 735/1479] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:08:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:08:13.226210 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44306fe-ef87-465d-b7ab-6a1b581841a4 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:08:13.226592 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44306fe-ef87-465d-b7ab-6a1b581841a4 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:08:13.226750 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44306fe-ef87-465d-b7ab-6a1b581841a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:08:13.291847 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-f44306fe-ef87-465d-b7ab-6a1b581841a4 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:08:13.293370 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f44306fe-ef87-465d-b7ab-6a1b581841a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:08:13.388936 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 745/1480] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:08:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:08:13.682207 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9537a0a6-d827-4535-98b2-44d508b98bc8 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:08:13.684001 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-9537a0a6-d827-4535-98b2-44d508b98bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:08:13.694901 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:08:13.827184 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9537a0a6-d827-4535-98b2-44d508b98bc8 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:08:13.905916 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:08:13.937080 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc877d1f-3ffe-4ab6-abd2-e036e788e08d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:08:13.937678 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc877d1f-3ffe-4ab6-abd2-e036e788e08d service nova] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:08:13.937950 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc877d1f-3ffe-4ab6-abd2-e036e788e08d service nova] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:08:13.938216 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc877d1f-3ffe-4ab6-abd2-e036e788e08d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:08:13.939832 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-cc877d1f-3ffe-4ab6-abd2-e036e788e08d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:08:14.026126 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc877d1f-3ffe-4ab6-abd2-e036e788e08d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:08:14.027729 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc877d1f-3ffe-4ab6-abd2-e036e788e08d service nova] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:08:14.044879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 736/1481] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:08:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:08:16.357394 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586270c3-2cf1-494f-a3f6-d4a964bf0f15 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:08:16.357959 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586270c3-2cf1-494f-a3f6-d4a964bf0f15 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:08:16.357959 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586270c3-2cf1-494f-a3f6-d4a964bf0f15 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:08:16.359331 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 746/1482] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:08:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:08:16.364457 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ecf66-51e8-4647-afe4-1bdadf2159fa None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:08:16.364867 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ecf66-51e8-4647-afe4-1bdadf2159fa None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:08:16.364995 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ecf66-51e8-4647-afe4-1bdadf2159fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:08:16.429520 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-813ecf66-51e8-4647-afe4-1bdadf2159fa None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:08:16.430713 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-813ecf66-51e8-4647-afe4-1bdadf2159fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:08:16.476202 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:08:16.526148 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 737/1483] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:08:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:08:16.812422 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586270c3-2cf1-494f-a3f6-d4a964bf0f15 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:08:16.813992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-586270c3-2cf1-494f-a3f6-d4a964bf0f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:08:16.837123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:08:16.914741 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586270c3-2cf1-494f-a3f6-d4a964bf0f15 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:08:16.985528 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de5bb28-b3f4-49c7-8d0a-40651c2c2a0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:08:16.986347 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de5bb28-b3f4-49c7-8d0a-40651c2c2a0c service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:08:16.986647 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de5bb28-b3f4-49c7-8d0a-40651c2c2a0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:08:16.986947 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de5bb28-b3f4-49c7-8d0a-40651c2c2a0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:08:16.988581 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-5de5bb28-b3f4-49c7-8d0a-40651c2c2a0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:08:17.059984 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:08:17.093860 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de5bb28-b3f4-49c7-8d0a-40651c2c2a0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:08:17.095308 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de5bb28-b3f4-49c7-8d0a-40651c2c2a0c service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:08:17.113874 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 747/1484] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:08:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:08:19.303853 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7d83c-f6b2-4034-a560-91b6d9de0224 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:08:19.304397 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7d83c-f6b2-4034-a560-91b6d9de0224 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:08:19.304397 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7d83c-f6b2-4034-a560-91b6d9de0224 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:08:19.305751 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 738/1485] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:08:19.311243 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d18b1c-dc50-42f3-9be9-d9c06742233c None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:08:19.311702 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d18b1c-dc50-42f3-9be9-d9c06742233c None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:08:19.311814 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d18b1c-dc50-42f3-9be9-d9c06742233c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:08:19.377785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-92d18b1c-dc50-42f3-9be9-d9c06742233c None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:08:19.379039 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-92d18b1c-dc50-42f3-9be9-d9c06742233c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:08:19.485141 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 748/1486] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:08:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:08:19.777286 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb7d83c-f6b2-4034-a560-91b6d9de0224 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:08:19.778878 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-2eb7d83c-f6b2-4034-a560-91b6d9de0224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:08:19.885520 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb7d83c-f6b2-4034-a560-91b6d9de0224 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:08:19.963293 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:08:19.964953 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe102b-8cd9-4194-9d21-5edea6f6e796 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:08:19.965566 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe102b-8cd9-4194-9d21-5edea6f6e796 service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:08:19.966029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe102b-8cd9-4194-9d21-5edea6f6e796 service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:08:19.966461 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe102b-8cd9-4194-9d21-5edea6f6e796 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:08:19.968283 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-87fe102b-8cd9-4194-9d21-5edea6f6e796 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:08:20.074384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fe102b-8cd9-4194-9d21-5edea6f6e796 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:08:20.075760 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fe102b-8cd9-4194-9d21-5edea6f6e796 service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:08:20.093563 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 739/1487] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:08:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:08:22.379302 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501ace2-6937-4c87-a6ea-7194d8f62a97 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:08:22.379931 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501ace2-6937-4c87-a6ea-7194d8f62a97 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:08:22.379931 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501ace2-6937-4c87-a6ea-7194d8f62a97 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:08:22.381249 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 749/1488] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:08:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:08:22.386428 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e2805-740f-414f-a959-d3adcab1bf5e None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:08:22.386919 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e2805-740f-414f-a959-d3adcab1bf5e None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:08:22.387034 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e2805-740f-414f-a959-d3adcab1bf5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:08:22.451228 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-2e0e2805-740f-414f-a959-d3adcab1bf5e None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:08:22.452420 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-2e0e2805-740f-414f-a959-d3adcab1bf5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:08:22.504011 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:08:22.548172 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 740/1489] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:08:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:08:22.837867 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c501ace2-6937-4c87-a6ea-7194d8f62a97 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:08:22.839479 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c501ace2-6937-4c87-a6ea-7194d8f62a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:08:22.876120 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:08:22.944123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c501ace2-6937-4c87-a6ea-7194d8f62a97 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:08:23.018047 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b2361e-8264-4a21-9a9d-8a7df2732b7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:08:23.018962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2361e-8264-4a21-9a9d-8a7df2732b7e service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:08:23.019240 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2361e-8264-4a21-9a9d-8a7df2732b7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:08:23.019503 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2361e-8264-4a21-9a9d-8a7df2732b7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:08:23.021070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-60b2361e-8264-4a21-9a9d-8a7df2732b7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:08:23.101033 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:08:23.129518 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b2361e-8264-4a21-9a9d-8a7df2732b7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:08:23.131462 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b2361e-8264-4a21-9a9d-8a7df2732b7e service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:08:23.149635 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 750/1490] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:08:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:08:25.442044 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6685f28f-e228-414e-bd93-18fcd867ba86 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:08:25.442890 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6685f28f-e228-414e-bd93-18fcd867ba86 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:08:25.442890 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6685f28f-e228-414e-bd93-18fcd867ba86 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:08:25.444261 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 741/1491] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 12 14:08:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:08:25.449438 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22eb8b-b3a7-4dfb-833d-106e349f0de8 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:08:25.449825 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22eb8b-b3a7-4dfb-833d-106e349f0de8 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:08:25.449988 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22eb8b-b3a7-4dfb-833d-106e349f0de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:08:25.515652 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-0b22eb8b-b3a7-4dfb-833d-106e349f0de8 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:08:25.516857 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-0b22eb8b-b3a7-4dfb-833d-106e349f0de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:08:25.612408 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 751/1492] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 12 14:08:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:08:25.897649 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6685f28f-e228-414e-bd93-18fcd867ba86 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:08:25.899223 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-6685f28f-e228-414e-bd93-18fcd867ba86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:08:26.005196 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6685f28f-e228-414e-bd93-18fcd867ba86 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:08:26.076305 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08549727-b1a7-4624-bac0-009531d73e56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:08:26.076902 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08549727-b1a7-4624-bac0-009531d73e56 service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:08:26.077178 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08549727-b1a7-4624-bac0-009531d73e56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:08:26.077446 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08549727-b1a7-4624-bac0-009531d73e56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:08:26.079067 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-08549727-b1a7-4624-bac0-009531d73e56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:08:26.087471 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:08:26.187484 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08549727-b1a7-4624-bac0-009531d73e56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:08:26.188890 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08549727-b1a7-4624-bac0-009531d73e56 service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:08:26.206475 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 742/1493] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:08:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:08:35.610825 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=82 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:08:36.204382 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=50 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:09:44.264561 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7d328-3fd6-4ed5-afd6-9beb24a869c3 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:09:44.265220 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7d328-3fd6-4ed5-afd6-9beb24a869c3 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:09:44.265220 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7d328-3fd6-4ed5-afd6-9beb24a869c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:09:44.342281 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-25f7d328-3fd6-4ed5-afd6-9beb24a869c3 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:09:44.343507 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-25f7d328-3fd6-4ed5-afd6-9beb24a869c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:09:44.435985 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 752/1494] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:09:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:09:44.444797 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a710fff-9c90-4a1e-8410-c3e0c7231546 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:09:44.445295 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a710fff-9c90-4a1e-8410-c3e0c7231546 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:09:44.445392 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a710fff-9c90-4a1e-8410-c3e0c7231546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:09:44.519456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-0a710fff-9c90-4a1e-8410-c3e0c7231546 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:09:44.520623 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-0a710fff-9c90-4a1e-8410-c3e0c7231546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:09:44.618675 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 743/1495] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:09:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:09:44.623470 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7d328-3fd6-4ed5-afd6-9beb24a869c3 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:09:44.624681 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-25f7d328-3fd6-4ed5-afd6-9beb24a869c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:09:44.635857 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7d328-3fd6-4ed5-afd6-9beb24a869c3 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:09:44.698186 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:09:44.730778 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e4f71-1d28-4afc-9c8f-2b76b804b37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:09:44.731402 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e4f71-1d28-4afc-9c8f-2b76b804b37b admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:09:44.731673 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e4f71-1d28-4afc-9c8f-2b76b804b37b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:09:44.731931 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e4f71-1d28-4afc-9c8f-2b76b804b37b admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:09:44.732563 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e4f71-1d28-4afc-9c8f-2b76b804b37b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:09:44.732854 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e4f71-1d28-4afc-9c8f-2b76b804b37b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:09:44.733850 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: WARNING py.warnings [None req-6e0e4f71-1d28-4afc-9c8f-2b76b804b37b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 12 14:09:44.733850 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: warnings.warn(msg) Sep 12 14:09:44.733850 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: Sep 12 14:09:44.734716 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e4f71-1d28-4afc-9c8f-2b76b804b37b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:09:44.746298 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 753/1496] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:09:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:09:44.754602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a710fff-9c90-4a1e-8410-c3e0c7231546 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:09:44.763435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a710fff-9c90-4a1e-8410-c3e0c7231546 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:09:44.822483 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:09:44.855336 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af1950b-ba66-41dd-9d71-c3922bb13422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:09:44.855935 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af1950b-ba66-41dd-9d71-c3922bb13422 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:09:44.856206 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af1950b-ba66-41dd-9d71-c3922bb13422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:09:44.856472 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af1950b-ba66-41dd-9d71-c3922bb13422 admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:09:44.857102 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af1950b-ba66-41dd-9d71-c3922bb13422 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:09:44.858805 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: WARNING py.warnings [None req-6af1950b-ba66-41dd-9d71-c3922bb13422 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 14:09:44.858805 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: warnings.warn(msg) Sep 12 14:09:44.858805 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: Sep 12 14:09:44.859988 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af1950b-ba66-41dd-9d71-c3922bb13422 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:09:44.915166 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 744/1497] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:09:44 2022] POST /identity/v3/projects => generated 379 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:09:44.919386 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e4f71-1d28-4afc-9c8f-2b76b804b37b admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:09:44.920935 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-6e0e4f71-1d28-4afc-9c8f-2b76b804b37b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:09:44.954183 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e4f71-1d28-4afc-9c8f-2b76b804b37b admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:09:45.045431 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cac167-6797-45c0-8d1b-30b1d9a37fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:09:45.046029 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cac167-6797-45c0-8d1b-30b1d9a37fb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:09:45.046309 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cac167-6797-45c0-8d1b-30b1d9a37fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:09:45.046581 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cac167-6797-45c0-8d1b-30b1d9a37fb6 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:09:45.047235 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cac167-6797-45c0-8d1b-30b1d9a37fb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:09:45.048241 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: WARNING py.warnings [None req-c5cac167-6797-45c0-8d1b-30b1d9a37fb6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 14:09:45.048241 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: warnings.warn(msg) Sep 12 14:09:45.048241 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: Sep 12 14:09:45.049438 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cac167-6797-45c0-8d1b-30b1d9a37fb6 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:09:45.090527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 754/1498] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:09:44 2022] POST /identity/v3/users => generated 290 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:09:45.094674 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af1950b-ba66-41dd-9d71-c3922bb13422 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:09:45.096216 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-6af1950b-ba66-41dd-9d71-c3922bb13422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:09:45.158587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af1950b-ba66-41dd-9d71-c3922bb13422 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:09:45.240554 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:09:45.251132 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6027525-2b8b-4e19-8817-ee15fc74aab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:09:45.251740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6027525-2b8b-4e19-8817-ee15fc74aab3 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:09:45.252015 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6027525-2b8b-4e19-8817-ee15fc74aab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:09:45.252285 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6027525-2b8b-4e19-8817-ee15fc74aab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:09:45.252922 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6027525-2b8b-4e19-8817-ee15fc74aab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:09:45.253921 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: WARNING py.warnings [None req-b6027525-2b8b-4e19-8817-ee15fc74aab3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 12 14:09:45.253921 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: warnings.warn(msg) Sep 12 14:09:45.253921 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: Sep 12 14:09:45.254804 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6027525-2b8b-4e19-8817-ee15fc74aab3 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:09:45.266878 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 745/1499] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:09:45 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:09:45.271365 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cac167-6797-45c0-8d1b-30b1d9a37fb6 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:09:45.272877 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c5cac167-6797-45c0-8d1b-30b1d9a37fb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:09:45.284069 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cac167-6797-45c0-8d1b-30b1d9a37fb6 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:09:45.309369 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:09:45.378594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a990fa-b74f-450c-9342-dd5220c86852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:09:45.379142 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a990fa-b74f-450c-9342-dd5220c86852 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:09:45.379416 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a990fa-b74f-450c-9342-dd5220c86852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:09:45.379681 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a990fa-b74f-450c-9342-dd5220c86852 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:09:45.380910 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a990fa-b74f-450c-9342-dd5220c86852 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:09:45.381948 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: WARNING py.warnings [None req-a1a990fa-b74f-450c-9342-dd5220c86852 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 12 14:09:45.381948 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: warnings.warn(msg) Sep 12 14:09:45.381948 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: Sep 12 14:09:45.382815 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a990fa-b74f-450c-9342-dd5220c86852 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:09:45.394756 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 755/1500] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:09:45 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:09:45.399699 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6027525-2b8b-4e19-8817-ee15fc74aab3 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:09:45.401206 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-b6027525-2b8b-4e19-8817-ee15fc74aab3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:09:45.433957 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6027525-2b8b-4e19-8817-ee15fc74aab3 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:09:45.526204 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5630b9d-7996-41ee-872e-f62354b493bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:09:45.526919 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5630b9d-7996-41ee-872e-f62354b493bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:09:45.527200 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5630b9d-7996-41ee-872e-f62354b493bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:09:45.527473 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5630b9d-7996-41ee-872e-f62354b493bf admin admin] PATH_INFO: `/v3/projects/097d89bc9aed4e9da94df92ba9d1da9e/users/0b2541f4ad4d42edb38e05debb5aa4e1/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:09:45.549725 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5630b9d-7996-41ee-872e-f62354b493bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=097d89bc9aed4e9da94df92ba9d1da9e, user_id=0b2541f4ad4d42edb38e05debb5aa4e1, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:09:45.550746 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: WARNING py.warnings [None req-a5630b9d-7996-41ee-872e-f62354b493bf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 14:09:45.550746 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: warnings.warn(msg) Sep 12 14:09:45.550746 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: Sep 12 14:09:45.552745 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5630b9d-7996-41ee-872e-f62354b493bf admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:09:45.576618 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 746/1501] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:09:45 2022] PUT /identity/v3/projects/097d89bc9aed4e9da94df92ba9d1da9e/users/0b2541f4ad4d42edb38e05debb5aa4e1/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:09:45.589347 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807177a3-fd5e-4220-a21f-d7c71379c393 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:09:45.589635 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807177a3-fd5e-4220-a21f-d7c71379c393 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:09:45.589909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807177a3-fd5e-4220-a21f-d7c71379c393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:09:45.646530 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-807177a3-fd5e-4220-a21f-d7c71379c393 None None] MFA Rules not processed for user `0b2541f4ad4d42edb38e05debb5aa4e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:09:45.647718 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-807177a3-fd5e-4220-a21f-d7c71379c393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:09:45.724634 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 756/1502] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:09:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:09:45.737604 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b3aeb-376a-4010-a82e-94e1a777b899 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:09:45.737885 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b3aeb-376a-4010-a82e-94e1a777b899 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:09:45.738156 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b3aeb-376a-4010-a82e-94e1a777b899 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:09:45.740082 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 747/1503] 10.222.0.42 () {54 vars in 964 bytes} [Mon Sep 12 14:09:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:09:45.744697 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141cffb0-39bf-4f61-a9fe-36b0c6c69a3b None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:09:45.745007 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141cffb0-39bf-4f61-a9fe-36b0c6c69a3b None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:09:45.745279 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141cffb0-39bf-4f61-a9fe-36b0c6c69a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:09:45.777664 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:09:45.821535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-141cffb0-39bf-4f61-a9fe-36b0c6c69a3b None None] MFA Rules not processed for user `8da38bc8dd2f4cf798649ce96097a4b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:09:45.822739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-141cffb0-39bf-4f61-a9fe-36b0c6c69a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:09:45.896026 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 757/1504] 10.222.0.42 () {58 vars in 1061 bytes} [Mon Sep 12 14:09:45 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:09:45.903892 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b3aeb-376a-4010-a82e-94e1a777b899 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:09:45.905407 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-344b3aeb-376a-4010-a82e-94e1a777b899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:09:45.955120 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:09:46.008861 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b3aeb-376a-4010-a82e-94e1a777b899 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:09:46.080638 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91161b4c-8633-40d2-a562-16637cc64f63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:09:46.081235 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91161b4c-8633-40d2-a562-16637cc64f63 service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:09:46.081509 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91161b4c-8633-40d2-a562-16637cc64f63 service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:09:46.081773 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91161b4c-8633-40d2-a562-16637cc64f63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:09:46.083408 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-91161b4c-8633-40d2-a562-16637cc64f63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:09:46.180101 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91161b4c-8633-40d2-a562-16637cc64f63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:09:46.181422 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91161b4c-8633-40d2-a562-16637cc64f63 service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:09:46.184822 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:09:46.199599 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 748/1505] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:09:45 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:09:46.244480 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5a00c-1706-4d3a-8e6f-581295a17f7e None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:09:46.244766 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5a00c-1706-4d3a-8e6f-581295a17f7e None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:09:46.245036 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5a00c-1706-4d3a-8e6f-581295a17f7e None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:09:46.246312 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 758/1506] 10.222.0.42 () {52 vars in 830 bytes} [Mon Sep 12 14:09:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:09:46.250034 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91161b4c-8633-40d2-a562-16637cc64f63 service cinder] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:09:46.251301 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-91161b4c-8633-40d2-a562-16637cc64f63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:09:46.347259 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91161b4c-8633-40d2-a562-16637cc64f63 service cinder] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:09:46.438446 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924c3d2a-84b2-4ee1-bfad-5a8046630c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:09:46.439036 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c3d2a-84b2-4ee1-bfad-5a8046630c61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:09:46.439313 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c3d2a-84b2-4ee1-bfad-5a8046630c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:09:46.439607 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c3d2a-84b2-4ee1-bfad-5a8046630c61 admin admin] PATH_INFO: `/v3/users/0b2541f4ad4d42edb38e05debb5aa4e1` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:09:46.452288 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924c3d2a-84b2-4ee1-bfad-5a8046630c61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b2541f4ad4d42edb38e05debb5aa4e1)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:09:46.453305 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: WARNING py.warnings [None req-924c3d2a-84b2-4ee1-bfad-5a8046630c61 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 14:09:46.453305 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: warnings.warn(msg) Sep 12 14:09:46.453305 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: Sep 12 14:09:46.454491 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924c3d2a-84b2-4ee1-bfad-5a8046630c61 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:09:46.464375 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:09:46.518436 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-924c3d2a-84b2-4ee1-bfad-5a8046630c61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b2541f4ad4d42edb38e05debb5aa4e1'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:09:46.527749 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-924c3d2a-84b2-4ee1-bfad-5a8046630c61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b2541f4ad4d42edb38e05debb5aa4e1'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:09:46.528314 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-924c3d2a-84b2-4ee1-bfad-5a8046630c61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b2541f4ad4d42edb38e05debb5aa4e1'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:09:46.528659 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-924c3d2a-84b2-4ee1-bfad-5a8046630c61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b2541f4ad4d42edb38e05debb5aa4e1'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:09:46.543723 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-924c3d2a-84b2-4ee1-bfad-5a8046630c61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b2541f4ad4d42edb38e05debb5aa4e1'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:09:46.571486 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 749/1507] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:09:46 2022] DELETE /identity/v3/users/0b2541f4ad4d42edb38e05debb5aa4e1 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:09:46.576120 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2220d3-6855-4ca0-9ce6-2d6eddf6e46c None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:09:46.576561 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2220d3-6855-4ca0-9ce6-2d6eddf6e46c None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:09:46.576680 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2220d3-6855-4ca0-9ce6-2d6eddf6e46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:09:46.633094 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-ec2220d3-6855-4ca0-9ce6-2d6eddf6e46c None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:09:46.634288 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ec2220d3-6855-4ca0-9ce6-2d6eddf6e46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:09:46.710811 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:09:46.723255 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 759/1508] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:09:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:09:46.733295 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924c3d2a-84b2-4ee1-bfad-5a8046630c61 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:09:46.734563 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-924c3d2a-84b2-4ee1-bfad-5a8046630c61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:09:46.842587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924c3d2a-84b2-4ee1-bfad-5a8046630c61 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:09:46.896863 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:09:46.916680 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb23fb-8532-46e9-aa88-77b670a3a0ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:09:46.917282 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb23fb-8532-46e9-aa88-77b670a3a0ee service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:09:46.917555 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb23fb-8532-46e9-aa88-77b670a3a0ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:09:46.917821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb23fb-8532-46e9-aa88-77b670a3a0ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:09:46.919426 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-30cb23fb-8532-46e9-aa88-77b670a3a0ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:09:47.024041 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb23fb-8532-46e9-aa88-77b670a3a0ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:09:47.025447 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb23fb-8532-46e9-aa88-77b670a3a0ee service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:09:47.043093 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 750/1509] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:09:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:09:47.383684 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2220d3-6855-4ca0-9ce6-2d6eddf6e46c None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:09:47.384934 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ec2220d3-6855-4ca0-9ce6-2d6eddf6e46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:09:47.396185 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2220d3-6855-4ca0-9ce6-2d6eddf6e46c None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:09:47.493087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1361591-ed6e-4454-bcf2-27fbe58202a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:09:47.493706 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1361591-ed6e-4454-bcf2-27fbe58202a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:09:47.493978 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1361591-ed6e-4454-bcf2-27fbe58202a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:09:47.494241 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1361591-ed6e-4454-bcf2-27fbe58202a0 admin admin] PATH_INFO: `/v3/projects/097d89bc9aed4e9da94df92ba9d1da9e` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:09:47.503797 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1361591-ed6e-4454-bcf2-27fbe58202a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=097d89bc9aed4e9da94df92ba9d1da9e)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:09:47.504802 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: WARNING py.warnings [None req-d1361591-ed6e-4454-bcf2-27fbe58202a0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 14:09:47.504802 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: warnings.warn(msg) Sep 12 14:09:47.504802 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: Sep 12 14:09:47.506680 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1361591-ed6e-4454-bcf2-27fbe58202a0 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:09:47.562859 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d1361591-ed6e-4454-bcf2-27fbe58202a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '097d89bc9aed4e9da94df92ba9d1da9e', 'request_id': 'req-d1361591-ed6e-4454-bcf2-27fbe58202a0'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:09:47.572849 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d1361591-ed6e-4454-bcf2-27fbe58202a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '097d89bc9aed4e9da94df92ba9d1da9e', 'request_id': 'req-d1361591-ed6e-4454-bcf2-27fbe58202a0'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:09:47.579003 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d1361591-ed6e-4454-bcf2-27fbe58202a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '097d89bc9aed4e9da94df92ba9d1da9e', 'request_id': 'req-d1361591-ed6e-4454-bcf2-27fbe58202a0'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:09:47.608490 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d1361591-ed6e-4454-bcf2-27fbe58202a0 admin admin] The token cache is being invalidate because project 097d89bc9aed4e9da94df92ba9d1da9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:09:47.608776 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d1361591-ed6e-4454-bcf2-27fbe58202a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:09:47.609202 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d1361591-ed6e-4454-bcf2-27fbe58202a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:09:47.610586 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 760/1510] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:09:47 2022] DELETE /identity/v3/projects/097d89bc9aed4e9da94df92ba9d1da9e => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:09:57.040893 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=62 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:09:57.606497 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:11:12.987613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b792dfa5-ca78-49df-b7df-b9fae315c8fc None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:11:12.988192 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b792dfa5-ca78-49df-b7df-b9fae315c8fc None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:11:12.988192 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b792dfa5-ca78-49df-b7df-b9fae315c8fc None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:11:12.989491 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 751/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 14:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:11:12.994526 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524827ae-3e45-4253-ae53-732e18ca1de4 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:11:12.994978 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524827ae-3e45-4253-ae53-732e18ca1de4 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:11:12.995081 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524827ae-3e45-4253-ae53-732e18ca1de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:11:13.081045 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-524827ae-3e45-4253-ae53-732e18ca1de4 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:11:13.082257 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-524827ae-3e45-4253-ae53-732e18ca1de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:11:13.183576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 761/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 14:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:11:13.337015 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b792dfa5-ca78-49df-b7df-b9fae315c8fc None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:11:13.339148 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-b792dfa5-ca78-49df-b7df-b9fae315c8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:11:13.458902 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b792dfa5-ca78-49df-b7df-b9fae315c8fc None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:11:13.465069 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76091cf0-79ca-4e87-9f7c-468ccb223b86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:11:13.465674 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76091cf0-79ca-4e87-9f7c-468ccb223b86 service nova] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:11:13.465956 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76091cf0-79ca-4e87-9f7c-468ccb223b86 service nova] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:11:13.466244 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76091cf0-79ca-4e87-9f7c-468ccb223b86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:11:13.475688 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76091cf0-79ca-4e87-9f7c-468ccb223b86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:11:13.477257 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76091cf0-79ca-4e87-9f7c-468ccb223b86 service nova] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:11:13.491360 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 752/1513] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:11:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:11:14.572468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32904416-a460-4be4-ae6c-e0e259a22eb2 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:11:14.573071 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32904416-a460-4be4-ae6c-e0e259a22eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:11:14.573071 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32904416-a460-4be4-ae6c-e0e259a22eb2 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:11:14.574354 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 762/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 14:11:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 14:11:14.580571 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1b2b6-07e1-4afe-a94f-d26d9fd9e166 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:11:14.581036 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1b2b6-07e1-4afe-a94f-d26d9fd9e166 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:11:14.581153 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1b2b6-07e1-4afe-a94f-d26d9fd9e166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:11:14.600720 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-25c1b2b6-07e1-4afe-a94f-d26d9fd9e166 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:11:14.601911 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-25c1b2b6-07e1-4afe-a94f-d26d9fd9e166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:11:14.609667 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 753/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 14:11:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:11:14.760516 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32904416-a460-4be4-ae6c-e0e259a22eb2 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:11:14.774697 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32904416-a460-4be4-ae6c-e0e259a22eb2 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:11:14.780017 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7e843-1562-4628-8bf2-3d05d961a75c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:11:14.780605 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7e843-1562-4628-8bf2-3d05d961a75c service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:11:14.780883 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7e843-1562-4628-8bf2-3d05d961a75c service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:11:14.781154 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7e843-1562-4628-8bf2-3d05d961a75c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:11:14.789474 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad7e843-1562-4628-8bf2-3d05d961a75c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:11:14.791060 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad7e843-1562-4628-8bf2-3d05d961a75c service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:11:14.800724 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 763/1516] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:11:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:11:14.913151 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79285aaf-ae5b-49bd-9fe2-ac3973c3ae0f None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:11:14.913430 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79285aaf-ae5b-49bd-9fe2-ac3973c3ae0f None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:11:14.913711 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79285aaf-ae5b-49bd-9fe2-ac3973c3ae0f None None] PATH_INFO: `/v3` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:11:14.915010 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 754/1517] 10.222.0.23 () {54 vars in 925 bytes} [Mon Sep 12 14:11:14 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:11:14.941175 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ec8e7-d432-42bf-9592-fe603e3347db None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:11:14.941462 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ec8e7-d432-42bf-9592-fe603e3347db None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:11:14.941734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ec8e7-d432-42bf-9592-fe603e3347db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:11:15.005471 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-4c6ec8e7-d432-42bf-9592-fe603e3347db None None] MFA Rules not processed for user `ce508bb9aa164a7fb1831a9b5ff5b932`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:11:15.006667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-4c6ec8e7-d432-42bf-9592-fe603e3347db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:11:15.023753 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:11:15.077677 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 764/1518] 10.222.0.23 () {58 vars in 1015 bytes} [Mon Sep 12 14:11:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:11:18.661467 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19285a41-d8e2-4505-9ba5-ad3e4ac9a64e None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:11:18.662017 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19285a41-d8e2-4505-9ba5-ad3e4ac9a64e None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:11:18.662017 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19285a41-d8e2-4505-9ba5-ad3e4ac9a64e None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:11:18.663399 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 755/1519] 10.222.0.42 () {54 vars in 961 bytes} [Mon Sep 12 14:11:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:11:18.669221 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee172491-8526-4f35-b55b-b36aadde7470 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:11:18.669690 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee172491-8526-4f35-b55b-b36aadde7470 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:11:18.669786 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee172491-8526-4f35-b55b-b36aadde7470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:11:18.690529 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-ee172491-8526-4f35-b55b-b36aadde7470 None None] MFA Rules not processed for user `ce508bb9aa164a7fb1831a9b5ff5b932`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:11:18.692032 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ee172491-8526-4f35-b55b-b36aadde7470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:11:18.701205 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 765/1520] 10.222.0.42 () {58 vars in 1058 bytes} [Mon Sep 12 14:11:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:11:18.709494 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19285a41-d8e2-4505-9ba5-ad3e4ac9a64e None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:11:18.723249 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19285a41-d8e2-4505-9ba5-ad3e4ac9a64e None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:11:18.728218 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f724aadd-41b1-41c8-a399-ea701ec90382 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce508bb9aa164a7fb1831a9b5ff5b932', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:11:18.728803 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f724aadd-41b1-41c8-a399-ea701ec90382 service placement] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:11:18.729081 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f724aadd-41b1-41c8-a399-ea701ec90382 service placement] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:11:18.729354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f724aadd-41b1-41c8-a399-ea701ec90382 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:11:18.737959 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f724aadd-41b1-41c8-a399-ea701ec90382 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:11:18.739592 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f724aadd-41b1-41c8-a399-ea701ec90382 service placement] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:11:18.749283 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 756/1521] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:11:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:11:26.266283 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb5c123-252b-448e-a6b0-bbaa4c751256 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:11:26.266914 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb5c123-252b-448e-a6b0-bbaa4c751256 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:11:26.266914 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb5c123-252b-448e-a6b0-bbaa4c751256 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:11:26.268262 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 766/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 14:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:11:26.274343 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd344a-430b-4143-852c-15d175cf3717 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:11:26.274792 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd344a-430b-4143-852c-15d175cf3717 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:11:26.274958 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd344a-430b-4143-852c-15d175cf3717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:11:26.294739 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-7dbd344a-430b-4143-852c-15d175cf3717 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:11:26.295954 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-7dbd344a-430b-4143-852c-15d175cf3717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:11:26.304251 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 757/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 14:11:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:11:26.455889 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb5c123-252b-448e-a6b0-bbaa4c751256 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:11:26.470102 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb5c123-252b-448e-a6b0-bbaa4c751256 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:11:26.475560 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f568b-1fcb-417d-84b3-27f29cfba278 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:11:26.476155 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f568b-1fcb-417d-84b3-27f29cfba278 service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:11:26.476431 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f568b-1fcb-417d-84b3-27f29cfba278 service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:11:26.476699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f568b-1fcb-417d-84b3-27f29cfba278 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:11:26.484937 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f568b-1fcb-417d-84b3-27f29cfba278 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:11:26.486499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f568b-1fcb-417d-84b3-27f29cfba278 service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:11:26.496306 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 767/1524] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:11:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:11:27.564417 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9cb31-3c5d-4359-b69d-91b166fa9d9d None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:11:27.564995 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9cb31-3c5d-4359-b69d-91b166fa9d9d None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:11:27.564995 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9cb31-3c5d-4359-b69d-91b166fa9d9d None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:11:27.567117 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 758/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 14:11:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:11:27.573394 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7a5f5-1fb6-46b1-bb7c-0a8f1125d3a4 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:11:27.573840 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7a5f5-1fb6-46b1-bb7c-0a8f1125d3a4 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:11:27.573936 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7a5f5-1fb6-46b1-bb7c-0a8f1125d3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:11:27.594132 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-18f7a5f5-1fb6-46b1-bb7c-0a8f1125d3a4 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:11:27.595375 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-18f7a5f5-1fb6-46b1-bb7c-0a8f1125d3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:11:27.603435 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 768/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 14:11:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:11:27.754700 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af9cb31-3c5d-4359-b69d-91b166fa9d9d None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:11:27.768615 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af9cb31-3c5d-4359-b69d-91b166fa9d9d None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:11:27.773952 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d71b66c-0284-4514-ac8d-efbd49e3a2a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:11:27.774536 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d71b66c-0284-4514-ac8d-efbd49e3a2a2 service nova] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:11:27.774822 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d71b66c-0284-4514-ac8d-efbd49e3a2a2 service nova] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:11:27.775111 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d71b66c-0284-4514-ac8d-efbd49e3a2a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:11:27.782977 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d71b66c-0284-4514-ac8d-efbd49e3a2a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:11:27.784563 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d71b66c-0284-4514-ac8d-efbd49e3a2a2 service nova] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:11:27.793652 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 759/1527] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:11:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:11:31.749702 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b76271-7663-4603-b0bc-0c163c6cb53e None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:11:31.750237 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b76271-7663-4603-b0bc-0c163c6cb53e None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:11:31.750237 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b76271-7663-4603-b0bc-0c163c6cb53e None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:11:31.752162 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 769/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 14:11:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:11:31.757693 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400b255-4be8-412d-8d45-510850ed1f1c None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:11:31.758139 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400b255-4be8-412d-8d45-510850ed1f1c None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:11:31.758245 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400b255-4be8-412d-8d45-510850ed1f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:11:31.779851 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-7400b255-4be8-412d-8d45-510850ed1f1c None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:11:31.781033 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-7400b255-4be8-412d-8d45-510850ed1f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:11:31.788675 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 760/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 14:11:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:11:32.071756 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b76271-7663-4603-b0bc-0c163c6cb53e None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:11:32.073434 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-89b76271-7663-4603-b0bc-0c163c6cb53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:11:32.149149 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b76271-7663-4603-b0bc-0c163c6cb53e None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:11:32.152639 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7bada-6c83-4293-8554-5620e3dcdd55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:11:32.153230 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7bada-6c83-4293-8554-5620e3dcdd55 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:11:32.153506 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7bada-6c83-4293-8554-5620e3dcdd55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:11:32.153773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7bada-6c83-4293-8554-5620e3dcdd55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:11:32.162415 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde7bada-6c83-4293-8554-5620e3dcdd55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:11:32.163835 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde7bada-6c83-4293-8554-5620e3dcdd55 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:11:32.172971 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 770/1530] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:11:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:11:34.316383 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286732e-a9df-4c7f-b909-8ebeb69b8e07 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:11:34.316949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286732e-a9df-4c7f-b909-8ebeb69b8e07 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:11:34.316949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286732e-a9df-4c7f-b909-8ebeb69b8e07 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:11:34.318278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 761/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 14:11:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:11:34.323725 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d2ad8-e2f3-4b39-9571-f528a7d623d2 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:11:34.324163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d2ad8-e2f3-4b39-9571-f528a7d623d2 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:11:34.324262 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d2ad8-e2f3-4b39-9571-f528a7d623d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:11:34.343623 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-4a4d2ad8-e2f3-4b39-9571-f528a7d623d2 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:11:34.344812 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-4a4d2ad8-e2f3-4b39-9571-f528a7d623d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:11:34.353222 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 771/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 14:11:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:11:34.646708 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c286732e-a9df-4c7f-b909-8ebeb69b8e07 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:11:34.657272 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c286732e-a9df-4c7f-b909-8ebeb69b8e07 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:11:34.661372 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f1f0ce-56c1-4e55-9c71-a593190b9620 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:11:34.661957 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1f0ce-56c1-4e55-9c71-a593190b9620 service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:11:34.662227 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1f0ce-56c1-4e55-9c71-a593190b9620 service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:11:34.662492 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1f0ce-56c1-4e55-9c71-a593190b9620 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:11:34.671773 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f1f0ce-56c1-4e55-9c71-a593190b9620 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:11:34.673194 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f1f0ce-56c1-4e55-9c71-a593190b9620 service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:11:34.682794 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 762/1533] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:11:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:11:38.304019 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e733c-f6aa-4f0e-9a2a-024a6f1dfaae None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:11:38.304569 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e733c-f6aa-4f0e-9a2a-024a6f1dfaae None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:11:38.304676 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e733c-f6aa-4f0e-9a2a-024a6f1dfaae None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:11:38.305982 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 772/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 14:11:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:11:38.311033 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf27db-8e3e-4103-acd7-860934388578 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:11:38.311464 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf27db-8e3e-4103-acd7-860934388578 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:11:38.311569 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf27db-8e3e-4103-acd7-860934388578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:11:38.319692 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=81 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:11:38.332648 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-afbf27db-8e3e-4103-acd7-860934388578 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:11:38.333840 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-afbf27db-8e3e-4103-acd7-860934388578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:11:38.341655 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 763/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 14:11:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:11:38.641600 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e733c-f6aa-4f0e-9a2a-024a6f1dfaae None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:11:38.643878 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-790e733c-f6aa-4f0e-9a2a-024a6f1dfaae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:11:38.707631 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:11:38.721170 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e733c-f6aa-4f0e-9a2a-024a6f1dfaae None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:11:38.724699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7596325d-7363-4b10-acdc-a7b7af26a522 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:11:38.725296 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7596325d-7363-4b10-acdc-a7b7af26a522 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:11:38.725574 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7596325d-7363-4b10-acdc-a7b7af26a522 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:11:38.725845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7596325d-7363-4b10-acdc-a7b7af26a522 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:11:38.734567 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7596325d-7363-4b10-acdc-a7b7af26a522 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:11:38.736004 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7596325d-7363-4b10-acdc-a7b7af26a522 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:11:38.745172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 773/1536] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:11:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:11:40.921415 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198292c5-5194-4fca-8096-4200ac86c199 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:11:40.921970 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198292c5-5194-4fca-8096-4200ac86c199 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:11:40.921970 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198292c5-5194-4fca-8096-4200ac86c199 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:11:40.923326 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 764/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 14:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:11:40.929170 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462d6d3-012a-48a7-bd79-2c1fa71422db None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:11:40.929610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462d6d3-012a-48a7-bd79-2c1fa71422db None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:11:40.929898 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462d6d3-012a-48a7-bd79-2c1fa71422db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:11:40.950598 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-d462d6d3-012a-48a7-bd79-2c1fa71422db None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:11:40.951830 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d462d6d3-012a-48a7-bd79-2c1fa71422db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:11:40.959487 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 774/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 14:11:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:11:41.266287 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198292c5-5194-4fca-8096-4200ac86c199 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:11:41.268500 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-198292c5-5194-4fca-8096-4200ac86c199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:11:41.281167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198292c5-5194-4fca-8096-4200ac86c199 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:11:41.285254 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89cbe64-d543-4ed8-aa1e-b9f10deada43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:11:41.285840 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89cbe64-d543-4ed8-aa1e-b9f10deada43 service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:11:41.286116 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89cbe64-d543-4ed8-aa1e-b9f10deada43 service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:11:41.286385 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89cbe64-d543-4ed8-aa1e-b9f10deada43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:11:41.295025 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89cbe64-d543-4ed8-aa1e-b9f10deada43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:11:41.296424 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89cbe64-d543-4ed8-aa1e-b9f10deada43 service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:11:41.305563 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 765/1539] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:11:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:11:50.945590 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=11 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:11:51.301490 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=8 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:06.896235 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff19178b-24e7-48f9-9424-f8318a6e63ed None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:06.898185 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ff633-1b59-4936-a66b-5f47a7896e9b None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:06.898633 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ff633-1b59-4936-a66b-5f47a7896e9b None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:06.898737 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ff633-1b59-4936-a66b-5f47a7896e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:06.901949 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff19178b-24e7-48f9-9424-f8318a6e63ed None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:06.902283 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff19178b-24e7-48f9-9424-f8318a6e63ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:06.930216 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-993ff633-1b59-4936-a66b-5f47a7896e9b None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:06.936049 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-993ff633-1b59-4936-a66b-5f47a7896e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:06.936632 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-ff19178b-24e7-48f9-9424-f8318a6e63ed None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:06.937827 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ff19178b-24e7-48f9-9424-f8318a6e63ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:06.947440 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 766/1540] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:06.951985 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 775/1541] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:06.962013 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca51346-3315-43cf-9bc2-7013b66d3e97 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:06.962295 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca51346-3315-43cf-9bc2-7013b66d3e97 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:06.962557 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca51346-3315-43cf-9bc2-7013b66d3e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:06.978122 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71450c-d5c0-414e-b287-8a6a17d88d61 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:06.978438 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71450c-d5c0-414e-b287-8a6a17d88d61 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:06.978688 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71450c-d5c0-414e-b287-8a6a17d88d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:06.994977 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-5ca51346-3315-43cf-9bc2-7013b66d3e97 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:06.997673 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-5ca51346-3315-43cf-9bc2-7013b66d3e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:07.012055 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-7b71450c-d5c0-414e-b287-8a6a17d88d61 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:07.013277 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7b71450c-d5c0-414e-b287-8a6a17d88d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:07.017336 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 767/1542] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:07.021600 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca51346-3315-43cf-9bc2-7013b66d3e97 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:07.034674 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 776/1543] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:13:07.034888 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca51346-3315-43cf-9bc2-7013b66d3e97 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:07.038254 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c061630-71a6-4e6e-87f7-e819dfc9aaf4 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:07.038551 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c061630-71a6-4e6e-87f7-e819dfc9aaf4 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:07.038832 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c061630-71a6-4e6e-87f7-e819dfc9aaf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:07.045297 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba7acc6-372a-4cbd-8d75-e6e2da9a6aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:07.045926 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7acc6-372a-4cbd-8d75-e6e2da9a6aee admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:07.046199 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7acc6-372a-4cbd-8d75-e6e2da9a6aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:07.046468 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7acc6-372a-4cbd-8d75-e6e2da9a6aee admin admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:07.047210 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba7acc6-372a-4cbd-8d75-e6e2da9a6aee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:13:07.047619 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba7acc6-372a-4cbd-8d75-e6e2da9a6aee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:07.049854 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: WARNING py.warnings [None req-dba7acc6-372a-4cbd-8d75-e6e2da9a6aee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 12 14:13:07.049854 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: warnings.warn(msg) Sep 12 14:13:07.049854 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: Sep 12 14:13:07.050740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba7acc6-372a-4cbd-8d75-e6e2da9a6aee admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:07.075097 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 768/1544] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:13:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:13:07.075780 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-2c061630-71a6-4e6e-87f7-e819dfc9aaf4 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:07.076018 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba7acc6-372a-4cbd-8d75-e6e2da9a6aee admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:07.077021 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-2c061630-71a6-4e6e-87f7-e819dfc9aaf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:07.092455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba7acc6-372a-4cbd-8d75-e6e2da9a6aee admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:07.097157 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 777/1545] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:13:07.098622 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e315d-ee33-48d7-949c-0b17f1a9a1d2 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:07.098941 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e315d-ee33-48d7-949c-0b17f1a9a1d2 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:07.099322 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e315d-ee33-48d7-949c-0b17f1a9a1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:07.106163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb331ab-1e33-48eb-95c0-81610f8026b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:07.107008 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb331ab-1e33-48eb-95c0-81610f8026b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:07.107293 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb331ab-1e33-48eb-95c0-81610f8026b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:07.107562 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb331ab-1e33-48eb-95c0-81610f8026b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:07.108316 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb331ab-1e33-48eb-95c0-81610f8026b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:13:07.108621 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb331ab-1e33-48eb-95c0-81610f8026b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:07.110224 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb331ab-1e33-48eb-95c0-81610f8026b7 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:07.131558 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-7f6e315d-ee33-48d7-949c-0b17f1a9a1d2 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:07.132787 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7f6e315d-ee33-48d7-949c-0b17f1a9a1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:07.135130 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 769/1546] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:13:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:13:07.136029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb331ab-1e33-48eb-95c0-81610f8026b7 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:07.149980 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 778/1547] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:07.151486 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9103f1-8536-4e3c-b4fc-503da8c45982 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:07.151774 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9103f1-8536-4e3c-b4fc-503da8c45982 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:07.152051 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9103f1-8536-4e3c-b4fc-503da8c45982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:07.158147 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb331ab-1e33-48eb-95c0-81610f8026b7 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:07.168279 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2629c3-dcf2-423f-9445-3e575e8099ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:07.168903 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2629c3-dcf2-423f-9445-3e575e8099ce admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:07.169181 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2629c3-dcf2-423f-9445-3e575e8099ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:07.169466 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2629c3-dcf2-423f-9445-3e575e8099ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:07.170092 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2629c3-dcf2-423f-9445-3e575e8099ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:07.172030 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2629c3-dcf2-423f-9445-3e575e8099ce admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:07.186076 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-4e9103f1-8536-4e3c-b4fc-503da8c45982 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:07.187309 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-4e9103f1-8536-4e3c-b4fc-503da8c45982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:07.196835 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 779/1548] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:07.198046 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec3770-d6f6-4de4-a4d3-744a0da9d5a2 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:07.198328 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec3770-d6f6-4de4-a4d3-744a0da9d5a2 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:07.198597 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec3770-d6f6-4de4-a4d3-744a0da9d5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:07.219878 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 770/1549] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:13:07 2022] POST /identity/v3/projects => generated 373 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:07.219878 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2629c3-dcf2-423f-9445-3e575e8099ce admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:07.221710 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-b3ec3770-d6f6-4de4-a4d3-744a0da9d5a2 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:07.222930 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b3ec3770-d6f6-4de4-a4d3-744a0da9d5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:07.281326 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2629c3-dcf2-423f-9445-3e575e8099ce admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:07.282921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 780/1550] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:07.284952 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048095c6-86fe-4670-bdbf-5b638127f957 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:07.285244 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048095c6-86fe-4670-bdbf-5b638127f957 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:07.285518 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048095c6-86fe-4670-bdbf-5b638127f957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:07.287749 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db7ce8-9b40-470b-98f5-ac6f85d8890d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:07.288395 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db7ce8-9b40-470b-98f5-ac6f85d8890d admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:07.288632 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db7ce8-9b40-470b-98f5-ac6f85d8890d admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:07.288907 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db7ce8-9b40-470b-98f5-ac6f85d8890d admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:07.289539 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29db7ce8-9b40-470b-98f5-ac6f85d8890d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:07.291715 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29db7ce8-9b40-470b-98f5-ac6f85d8890d admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:07.305637 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-048095c6-86fe-4670-bdbf-5b638127f957 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:07.306828 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-048095c6-86fe-4670-bdbf-5b638127f957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:07.315617 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 781/1551] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:07.316795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857e521-cce9-4018-8a57-c36ebcb43e40 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:07.317085 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857e521-cce9-4018-8a57-c36ebcb43e40 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:07.317341 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857e521-cce9-4018-8a57-c36ebcb43e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:07.320223 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 771/1552] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:13:07 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:07.321473 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2094f-50e4-4c0f-b7be-14536c7447a6 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:07.321770 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2094f-50e4-4c0f-b7be-14536c7447a6 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:07.322048 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2094f-50e4-4c0f-b7be-14536c7447a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:07.337880 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-3857e521-cce9-4018-8a57-c36ebcb43e40 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:07.339124 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-3857e521-cce9-4018-8a57-c36ebcb43e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:07.342185 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-56d2094f-50e4-4c0f-b7be-14536c7447a6 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:07.343441 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-56d2094f-50e4-4c0f-b7be-14536c7447a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:07.398523 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 782/1553] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:07.400133 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c4452-fa5a-4ac2-a47a-01d0edd3d073 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:07.400419 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c4452-fa5a-4ac2-a47a-01d0edd3d073 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:07.401400 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c4452-fa5a-4ac2-a47a-01d0edd3d073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:07.410540 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 772/1554] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:07.411557 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2094f-50e4-4c0f-b7be-14536c7447a6 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:07.421035 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2094f-50e4-4c0f-b7be-14536c7447a6 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:07.422603 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-f28c4452-fa5a-4ac2-a47a-01d0edd3d073 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:07.423845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f28c4452-fa5a-4ac2-a47a-01d0edd3d073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:07.425796 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb6820-ae5b-4934-9b13-bba96dcc23fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:07.426424 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb6820-ae5b-4934-9b13-bba96dcc23fc admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:07.426704 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb6820-ae5b-4934-9b13-bba96dcc23fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:07.427008 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb6820-ae5b-4934-9b13-bba96dcc23fc admin admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:07.427652 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acb6820-ae5b-4934-9b13-bba96dcc23fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:13:07.427955 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acb6820-ae5b-4934-9b13-bba96dcc23fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:07.429550 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acb6820-ae5b-4934-9b13-bba96dcc23fc admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:07.431930 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 783/1555] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:07.432684 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c4452-fa5a-4ac2-a47a-01d0edd3d073 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:07.442102 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 773/1556] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:13:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:13:07.443528 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa1339-c768-482a-bfa1-eb26b859d1f1 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:07.443807 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa1339-c768-482a-bfa1-eb26b859d1f1 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:07.444198 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa1339-c768-482a-bfa1-eb26b859d1f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:07.448388 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c4452-fa5a-4ac2-a47a-01d0edd3d073 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:07.452444 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:07.453143 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f608f-9e14-4d83-86ee-5df78fbebfd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:07.454036 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f608f-9e14-4d83-86ee-5df78fbebfd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:07.454291 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f608f-9e14-4d83-86ee-5df78fbebfd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:07.454561 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f608f-9e14-4d83-86ee-5df78fbebfd4 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:07.455224 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f608f-9e14-4d83-86ee-5df78fbebfd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:07.457713 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f608f-9e14-4d83-86ee-5df78fbebfd4 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:07.467092 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-d3aa1339-c768-482a-bfa1-eb26b859d1f1 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:07.468276 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-d3aa1339-c768-482a-bfa1-eb26b859d1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:07.478604 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 774/1557] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:07.479410 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aa1339-c768-482a-bfa1-eb26b859d1f1 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:07.479980 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 784/1558] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:13:07 2022] POST /identity/v3/users => generated 287 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:07.480748 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f608f-9e14-4d83-86ee-5df78fbebfd4 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:07.490574 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f608f-9e14-4d83-86ee-5df78fbebfd4 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:07.493361 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aa1339-c768-482a-bfa1-eb26b859d1f1 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:07.497774 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f521115-3fd6-4e48-9117-ee6805a8f6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:07.498034 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d40ec49-2350-43d4-8435-c2a6e89e9bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:07.498367 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f521115-3fd6-4e48-9117-ee6805a8f6e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:07.498647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f521115-3fd6-4e48-9117-ee6805a8f6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:07.498952 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f521115-3fd6-4e48-9117-ee6805a8f6e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:07.499084 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d40ec49-2350-43d4-8435-c2a6e89e9bf3 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:07.499582 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d40ec49-2350-43d4-8435-c2a6e89e9bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:07.499700 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f521115-3fd6-4e48-9117-ee6805a8f6e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:13:07.499875 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f521115-3fd6-4e48-9117-ee6805a8f6e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:07.500089 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d40ec49-2350-43d4-8435-c2a6e89e9bf3 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:07.500953 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d40ec49-2350-43d4-8435-c2a6e89e9bf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:07.501470 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f521115-3fd6-4e48-9117-ee6805a8f6e3 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:07.502866 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d40ec49-2350-43d4-8435-c2a6e89e9bf3 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:07.514098 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 775/1559] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:13:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:13:07.515348 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220483a-b706-4a9d-b00a-ba5fd3d9b8f8 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:07.515647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220483a-b706-4a9d-b00a-ba5fd3d9b8f8 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:07.515894 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220483a-b706-4a9d-b00a-ba5fd3d9b8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:07.540117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 785/1560] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:13:07 2022] POST /identity/v3/users => generated 289 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:07.540117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b6287-9a47-4438-8c66-2614dd94f60a None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:07.540117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b6287-9a47-4438-8c66-2614dd94f60a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:07.540117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b6287-9a47-4438-8c66-2614dd94f60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:07.543953 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-7220483a-b706-4a9d-b00a-ba5fd3d9b8f8 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:07.545271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-7220483a-b706-4a9d-b00a-ba5fd3d9b8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:07.561442 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 776/1561] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:07.561442 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100590c8-8681-4157-a651-00a2a7fce3ae None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:07.561442 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100590c8-8681-4157-a651-00a2a7fce3ae None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:07.561442 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100590c8-8681-4157-a651-00a2a7fce3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:07.561883 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-de1b6287-9a47-4438-8c66-2614dd94f60a None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:07.561883 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-de1b6287-9a47-4438-8c66-2614dd94f60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:07.572653 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 786/1562] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:07.573646 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b6287-9a47-4438-8c66-2614dd94f60a None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:07.584912 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-100590c8-8681-4157-a651-00a2a7fce3ae None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:07.585084 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b6287-9a47-4438-8c66-2614dd94f60a None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:07.586093 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-100590c8-8681-4157-a651-00a2a7fce3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:07.589646 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27120e2a-d465-45c6-86ec-7744136f87ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:07.590257 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27120e2a-d465-45c6-86ec-7744136f87ba admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:07.590539 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27120e2a-d465-45c6-86ec-7744136f87ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:07.590811 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27120e2a-d465-45c6-86ec-7744136f87ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:07.591465 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27120e2a-d465-45c6-86ec-7744136f87ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:07.592456 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: WARNING py.warnings [None req-27120e2a-d465-45c6-86ec-7744136f87ba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 14:13:07.592456 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: warnings.warn(msg) Sep 12 14:13:07.592456 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: Sep 12 14:13:07.593683 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27120e2a-d465-45c6-86ec-7744136f87ba admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:07.594023 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 777/1563] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:07.594800 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100590c8-8681-4157-a651-00a2a7fce3ae None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:07.605557 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:07.605939 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100590c8-8681-4157-a651-00a2a7fce3ae None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:07.610759 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f8cb3-2658-4b67-8c66-2865e9aba9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:07.611482 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f8cb3-2658-4b67-8c66-2865e9aba9c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:07.611914 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f8cb3-2658-4b67-8c66-2865e9aba9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:07.612189 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f8cb3-2658-4b67-8c66-2865e9aba9c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:07.612908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f8cb3-2658-4b67-8c66-2865e9aba9c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:13:07.613187 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f8cb3-2658-4b67-8c66-2865e9aba9c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:07.614851 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f8cb3-2658-4b67-8c66-2865e9aba9c8 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:07.625310 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 787/1564] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:13:07 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:07.625310 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27120e2a-d465-45c6-86ec-7744136f87ba admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:07.630410 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 778/1565] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:13:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:13:07.634969 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f8cb3-2658-4b67-8c66-2865e9aba9c8 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:07.641223 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f8cb3-2658-4b67-8c66-2865e9aba9c8 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:07.694918 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27120e2a-d465-45c6-86ec-7744136f87ba admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:07.699456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25617a6e-9e07-4f94-8c31-17bf716a01b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:07.699619 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8009037a-a726-4fb8-b156-cb7b3303391e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:07.700058 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25617a6e-9e07-4f94-8c31-17bf716a01b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:07.700171 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8009037a-a726-4fb8-b156-cb7b3303391e admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:07.700331 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25617a6e-9e07-4f94-8c31-17bf716a01b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:07.700431 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8009037a-a726-4fb8-b156-cb7b3303391e admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:07.700598 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25617a6e-9e07-4f94-8c31-17bf716a01b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:07.700697 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8009037a-a726-4fb8-b156-cb7b3303391e admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:07.701235 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8009037a-a726-4fb8-b156-cb7b3303391e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:07.701342 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25617a6e-9e07-4f94-8c31-17bf716a01b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:07.702814 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8009037a-a726-4fb8-b156-cb7b3303391e admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:07.703265 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25617a6e-9e07-4f94-8c31-17bf716a01b5 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:07.713788 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 788/1566] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:07.714392 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8009037a-a726-4fb8-b156-cb7b3303391e admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:07.724962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8009037a-a726-4fb8-b156-cb7b3303391e admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:07.729781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92555b33-149d-43c2-bcec-964c8cd0dffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:07.730371 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92555b33-149d-43c2-bcec-964c8cd0dffa admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:07.730656 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92555b33-149d-43c2-bcec-964c8cd0dffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:07.730803 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 779/1567] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:13:07 2022] POST /identity/v3/projects => generated 405 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:07.730986 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92555b33-149d-43c2-bcec-964c8cd0dffa admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:07.731603 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92555b33-149d-43c2-bcec-964c8cd0dffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:07.731725 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25617a6e-9e07-4f94-8c31-17bf716a01b5 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:07.733227 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92555b33-149d-43c2-bcec-964c8cd0dffa admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:07.743415 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 789/1568] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:07.744241 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92555b33-149d-43c2-bcec-964c8cd0dffa admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:07.753892 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92555b33-149d-43c2-bcec-964c8cd0dffa admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:07.791648 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25617a6e-9e07-4f94-8c31-17bf716a01b5 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:07.796188 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c71fe6-81e2-4c98-85e2-e7de1abd9d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:07.796789 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c71fe6-81e2-4c98-85e2-e7de1abd9d0c admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:07.797084 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c71fe6-81e2-4c98-85e2-e7de1abd9d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:07.797356 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c71fe6-81e2-4c98-85e2-e7de1abd9d0c admin admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:07.797991 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c71fe6-81e2-4c98-85e2-e7de1abd9d0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:13:07.798288 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c71fe6-81e2-4c98-85e2-e7de1abd9d0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:07.799932 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c71fe6-81e2-4c98-85e2-e7de1abd9d0c admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:07.809796 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd9a05f-6817-4db9-b37d-a6f8264539ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:07.810398 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9a05f-6817-4db9-b37d-a6f8264539ad admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:07.810685 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9a05f-6817-4db9-b37d-a6f8264539ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:07.810993 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9a05f-6817-4db9-b37d-a6f8264539ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:07.811635 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd9a05f-6817-4db9-b37d-a6f8264539ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:13:07.816855 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 780/1569] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:13:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:13:07.817002 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd9a05f-6817-4db9-b37d-a6f8264539ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:07.817100 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c71fe6-81e2-4c98-85e2-e7de1abd9d0c admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:07.817192 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd9a05f-6817-4db9-b37d-a6f8264539ad admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:07.825123 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c71fe6-81e2-4c98-85e2-e7de1abd9d0c admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:07.835002 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 790/1570] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:13:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:13:07.835002 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd9a05f-6817-4db9-b37d-a6f8264539ad admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:07.835568 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf2ad26-de3a-4d98-8c3e-6a5ed34a53aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:07.836181 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf2ad26-de3a-4d98-8c3e-6a5ed34a53aa admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:07.836459 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf2ad26-de3a-4d98-8c3e-6a5ed34a53aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:07.836728 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf2ad26-de3a-4d98-8c3e-6a5ed34a53aa admin admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:07.837398 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf2ad26-de3a-4d98-8c3e-6a5ed34a53aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:13:07.837699 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf2ad26-de3a-4d98-8c3e-6a5ed34a53aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:07.839327 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf2ad26-de3a-4d98-8c3e-6a5ed34a53aa admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:07.845105 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd9a05f-6817-4db9-b37d-a6f8264539ad admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:07.849664 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f75de-feef-4f62-94a1-e48aa9361ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:07.850400 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f75de-feef-4f62-94a1-e48aa9361ee1 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:07.850673 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f75de-feef-4f62-94a1-e48aa9361ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:07.850972 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f75de-feef-4f62-94a1-e48aa9361ee1 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:07.851626 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f75de-feef-4f62-94a1-e48aa9361ee1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:07.859111 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 781/1571] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:13:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:13:07.859111 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf2ad26-de3a-4d98-8c3e-6a5ed34a53aa admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:07.859360 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f75de-feef-4f62-94a1-e48aa9361ee1 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:07.862573 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf2ad26-de3a-4d98-8c3e-6a5ed34a53aa admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:07.867637 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1ed60-5f6f-4943-be89-d42dd669057f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:07.868328 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1ed60-5f6f-4943-be89-d42dd669057f admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:07.868643 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1ed60-5f6f-4943-be89-d42dd669057f admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:07.868947 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1ed60-5f6f-4943-be89-d42dd669057f admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:07.869629 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e1ed60-5f6f-4943-be89-d42dd669057f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:07.871740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e1ed60-5f6f-4943-be89-d42dd669057f admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:07.879727 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 791/1572] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:13:07 2022] POST /identity/v3/users => generated 294 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:07.879727 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f75de-feef-4f62-94a1-e48aa9361ee1 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:07.887955 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f75de-feef-4f62-94a1-e48aa9361ee1 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:07.893465 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:07.893722 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e9255-0ece-4e86-af7d-ac751f61187d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:07.894347 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e9255-0ece-4e86-af7d-ac751f61187d admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:07.894624 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e9255-0ece-4e86-af7d-ac751f61187d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:07.894978 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e9255-0ece-4e86-af7d-ac751f61187d admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:07.895772 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92e9255-0ece-4e86-af7d-ac751f61187d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:07.897335 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92e9255-0ece-4e86-af7d-ac751f61187d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:07.904059 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 782/1573] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:13:07 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:07.904059 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1ed60-5f6f-4943-be89-d42dd669057f admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:07.910962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 792/1574] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:07.910962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e9255-0ece-4e86-af7d-ac751f61187d admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:07.918029 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e9255-0ece-4e86-af7d-ac751f61187d admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:07.958526 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:07.967041 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1ed60-5f6f-4943-be89-d42dd669057f admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:07.971744 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13fe9d1-b99c-4b8f-a76c-dff1c56a16ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:07.972459 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13fe9d1-b99c-4b8f-a76c-dff1c56a16ba admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:07.972767 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13fe9d1-b99c-4b8f-a76c-dff1c56a16ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:07.973039 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13fe9d1-b99c-4b8f-a76c-dff1c56a16ba admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:07.973665 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13fe9d1-b99c-4b8f-a76c-dff1c56a16ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:07.974681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: WARNING py.warnings [None req-e13fe9d1-b99c-4b8f-a76c-dff1c56a16ba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 14:13:07.974681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: warnings.warn(msg) Sep 12 14:13:07.974681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: Sep 12 14:13:07.975925 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13fe9d1-b99c-4b8f-a76c-dff1c56a16ba admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:07.976049 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0ef4e-b8c8-4332-804c-db8015d0fd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:07.976754 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0ef4e-b8c8-4332-804c-db8015d0fd86 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:07.977124 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0ef4e-b8c8-4332-804c-db8015d0fd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:07.977487 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0ef4e-b8c8-4332-804c-db8015d0fd86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:07.978212 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae0ef4e-b8c8-4332-804c-db8015d0fd86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:07.979892 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae0ef4e-b8c8-4332-804c-db8015d0fd86 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:07.990415 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 793/1575] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:07.991382 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0ef4e-b8c8-4332-804c-db8015d0fd86 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:08.004612 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 783/1576] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:13:07 2022] POST /identity/v3/users => generated 303 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:08.004786 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0ef4e-b8c8-4332-804c-db8015d0fd86 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:08.004935 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13fe9d1-b99c-4b8f-a76c-dff1c56a16ba admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:08.006796 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c198cb-4ffd-461c-ab23-e75f15a49fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:08.007410 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c198cb-4ffd-461c-ab23-e75f15a49fa0 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:08.007683 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c198cb-4ffd-461c-ab23-e75f15a49fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:08.007950 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c198cb-4ffd-461c-ab23-e75f15a49fa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:08.008568 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c198cb-4ffd-461c-ab23-e75f15a49fa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:08.010451 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c198cb-4ffd-461c-ab23-e75f15a49fa0 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:08.013908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13fe9d1-b99c-4b8f-a76c-dff1c56a16ba admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:08.019153 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15824364-8b6b-459d-9bce-e5a924837eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:08.019830 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15824364-8b6b-459d-9bce-e5a924837eb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:08.020169 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15824364-8b6b-459d-9bce-e5a924837eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:08.020442 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15824364-8b6b-459d-9bce-e5a924837eb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:08.021065 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15824364-8b6b-459d-9bce-e5a924837eb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:08.022966 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15824364-8b6b-459d-9bce-e5a924837eb2 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:08.037773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 794/1577] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:13:07 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:08.038465 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c198cb-4ffd-461c-ab23-e75f15a49fa0 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:08.051442 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 784/1578] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:13:08 2022] POST /identity/v3/projects => generated 389 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:08.055634 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15824364-8b6b-459d-9bce-e5a924837eb2 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:08.102247 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c198cb-4ffd-461c-ab23-e75f15a49fa0 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:08.122044 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15824364-8b6b-459d-9bce-e5a924837eb2 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:08.128987 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abedf3fd-7c4d-4be1-ba24-fa1431c244ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:08.129781 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abedf3fd-7c4d-4be1-ba24-fa1431c244ff admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:08.129991 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abedf3fd-7c4d-4be1-ba24-fa1431c244ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:08.130322 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abedf3fd-7c4d-4be1-ba24-fa1431c244ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:08.131067 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abedf3fd-7c4d-4be1-ba24-fa1431c244ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:08.132723 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abedf3fd-7c4d-4be1-ba24-fa1431c244ff admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:08.143057 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 785/1579] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:08.143057 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abedf3fd-7c4d-4be1-ba24-fa1431c244ff admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:08.153204 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abedf3fd-7c4d-4be1-ba24-fa1431c244ff admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:08.157746 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026db490-22bf-4a53-8566-4316b40a844d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:08.158318 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026db490-22bf-4a53-8566-4316b40a844d admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:08.158627 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026db490-22bf-4a53-8566-4316b40a844d admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:08.158923 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026db490-22bf-4a53-8566-4316b40a844d admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:08.159538 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026db490-22bf-4a53-8566-4316b40a844d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:08.161408 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026db490-22bf-4a53-8566-4316b40a844d admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:08.162420 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de82a27-3d42-4fcd-9ec3-483d5cee1bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:08.163215 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de82a27-3d42-4fcd-9ec3-483d5cee1bfe admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:08.163575 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de82a27-3d42-4fcd-9ec3-483d5cee1bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:08.163850 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de82a27-3d42-4fcd-9ec3-483d5cee1bfe admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:08.164484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de82a27-3d42-4fcd-9ec3-483d5cee1bfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:08.166378 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de82a27-3d42-4fcd-9ec3-483d5cee1bfe admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:08.181969 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 786/1580] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:13:08 2022] POST /identity/v3/users => generated 292 bytes in 38 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:08.182600 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026db490-22bf-4a53-8566-4316b40a844d admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:08.194740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026db490-22bf-4a53-8566-4316b40a844d admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:08.199941 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 795/1581] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:13:08 2022] POST /identity/v3/projects => generated 355 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:08.199941 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de82a27-3d42-4fcd-9ec3-483d5cee1bfe admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:08.238235 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:08.245530 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:08.253200 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4968f43-5fde-4f15-ae17-6d1618e1f660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:08.253879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4968f43-5fde-4f15-ae17-6d1618e1f660 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:08.254151 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4968f43-5fde-4f15-ae17-6d1618e1f660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:08.254416 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4968f43-5fde-4f15-ae17-6d1618e1f660 admin admin] PATH_INFO: `/v3/projects/cc1e89bb0f744de5b1c520d488ae1b25/users/de2460c2fdb04a949046acd6a3c0abaa/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:08.260923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de82a27-3d42-4fcd-9ec3-483d5cee1bfe admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:08.267354 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0704143-64ea-49e6-9ad6-2f02b5c76a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:08.268043 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0704143-64ea-49e6-9ad6-2f02b5c76a24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:08.268318 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0704143-64ea-49e6-9ad6-2f02b5c76a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:08.268592 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0704143-64ea-49e6-9ad6-2f02b5c76a24 admin admin] PATH_INFO: `/v3/projects/613c135e354b4262865198bd5c834cb1/users/e7cb679ed52e42ca83a2084f1ad4bd92/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:08.269466 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4968f43-5fde-4f15-ae17-6d1618e1f660 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc1e89bb0f744de5b1c520d488ae1b25, user_id=de2460c2fdb04a949046acd6a3c0abaa, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:08.272173 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4968f43-5fde-4f15-ae17-6d1618e1f660 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:08.286242 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0704143-64ea-49e6-9ad6-2f02b5c76a24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=613c135e354b4262865198bd5c834cb1, user_id=e7cb679ed52e42ca83a2084f1ad4bd92, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:08.287367 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: WARNING py.warnings [None req-d0704143-64ea-49e6-9ad6-2f02b5c76a24 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 14:13:08.287367 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: warnings.warn(msg) Sep 12 14:13:08.287367 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: Sep 12 14:13:08.287821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 787/1582] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:13:08 2022] PUT /identity/v3/projects/cc1e89bb0f744de5b1c520d488ae1b25/users/de2460c2fdb04a949046acd6a3c0abaa/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:08.288663 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4968f43-5fde-4f15-ae17-6d1618e1f660 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:08.289793 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0704143-64ea-49e6-9ad6-2f02b5c76a24 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:08.306127 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 796/1583] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:13:08 2022] PUT /identity/v3/projects/613c135e354b4262865198bd5c834cb1/users/e7cb679ed52e42ca83a2084f1ad4bd92/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:08.306962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0704143-64ea-49e6-9ad6-2f02b5c76a24 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:08.363028 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4968f43-5fde-4f15-ae17-6d1618e1f660 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:08.367681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1e89e8-960d-4a1c-ad4d-526bfc7d825f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:08.368282 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e89e8-960d-4a1c-ad4d-526bfc7d825f admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:08.368556 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e89e8-960d-4a1c-ad4d-526bfc7d825f admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:08.368792 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0704143-64ea-49e6-9ad6-2f02b5c76a24 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:08.368894 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e89e8-960d-4a1c-ad4d-526bfc7d825f admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:08.369440 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1e89e8-960d-4a1c-ad4d-526bfc7d825f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:08.371052 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1e89e8-960d-4a1c-ad4d-526bfc7d825f admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:08.373792 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11096381-02e4-4e52-8038-863a872a4044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:08.374419 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11096381-02e4-4e52-8038-863a872a4044 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:08.374697 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11096381-02e4-4e52-8038-863a872a4044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:08.375055 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11096381-02e4-4e52-8038-863a872a4044 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:08.375684 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11096381-02e4-4e52-8038-863a872a4044 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:08.378134 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11096381-02e4-4e52-8038-863a872a4044 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:08.381385 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 788/1584] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:08.382145 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1e89e8-960d-4a1c-ad4d-526bfc7d825f admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:08.392516 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1e89e8-960d-4a1c-ad4d-526bfc7d825f admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:08.397901 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0b010-0045-4f25-acb0-8ccf204e9d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:08.398598 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0b010-0045-4f25-acb0-8ccf204e9d05 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:08.398920 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0b010-0045-4f25-acb0-8ccf204e9d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:08.399195 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0b010-0045-4f25-acb0-8ccf204e9d05 admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:08.399849 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0b010-0045-4f25-acb0-8ccf204e9d05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:08.403185 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 797/1585] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:13:08 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:08.403185 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11096381-02e4-4e52-8038-863a872a4044 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:08.403408 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0b010-0045-4f25-acb0-8ccf204e9d05 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:08.411056 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11096381-02e4-4e52-8038-863a872a4044 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:08.415757 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad93def-d771-4e0d-a75a-b44bfb6a3ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:08.416398 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad93def-d771-4e0d-a75a-b44bfb6a3ba7 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:08.416617 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad93def-d771-4e0d-a75a-b44bfb6a3ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:08.416931 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad93def-d771-4e0d-a75a-b44bfb6a3ba7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:08.417683 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad93def-d771-4e0d-a75a-b44bfb6a3ba7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:08.419699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad93def-d771-4e0d-a75a-b44bfb6a3ba7 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:08.427440 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 789/1586] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:13:08 2022] POST /identity/v3/users => generated 295 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:08.428338 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0b010-0045-4f25-acb0-8ccf204e9d05 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:08.431355 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 798/1587] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:08.432133 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad93def-d771-4e0d-a75a-b44bfb6a3ba7 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:08.438568 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0b010-0045-4f25-acb0-8ccf204e9d05 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:08.441653 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad93def-d771-4e0d-a75a-b44bfb6a3ba7 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:08.446091 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74254835-4320-413a-b2b3-75c44768b562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:08.446664 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74254835-4320-413a-b2b3-75c44768b562 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:08.446975 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74254835-4320-413a-b2b3-75c44768b562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:08.447241 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74254835-4320-413a-b2b3-75c44768b562 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:08.447853 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74254835-4320-413a-b2b3-75c44768b562 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:08.449722 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74254835-4320-413a-b2b3-75c44768b562 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:08.450728 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3522fe87-e537-48cf-8577-391ef5dc28a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:08.451364 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3522fe87-e537-48cf-8577-391ef5dc28a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:08.451646 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3522fe87-e537-48cf-8577-391ef5dc28a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:08.451923 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3522fe87-e537-48cf-8577-391ef5dc28a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:08.452589 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3522fe87-e537-48cf-8577-391ef5dc28a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:08.454153 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3522fe87-e537-48cf-8577-391ef5dc28a1 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:08.465602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 790/1588] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:08.467055 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e3c9a-144a-4435-ad9e-1d461e9bc58d None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:08.467438 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e3c9a-144a-4435-ad9e-1d461e9bc58d None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:08.467827 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e3c9a-144a-4435-ad9e-1d461e9bc58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:08.473252 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 799/1589] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:13:08 2022] POST /identity/v3/users => generated 278 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:08.474382 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a31c8-f3b8-4ae6-8a7f-6773ab72c0c2 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:08.474664 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a31c8-f3b8-4ae6-8a7f-6773ab72c0c2 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:08.474964 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a31c8-f3b8-4ae6-8a7f-6773ab72c0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:08.494916 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-f06a31c8-f3b8-4ae6-8a7f-6773ab72c0c2 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:08.496114 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f06a31c8-f3b8-4ae6-8a7f-6773ab72c0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:08.497937 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-ca2e3c9a-144a-4435-ad9e-1d461e9bc58d None None] MFA Rules not processed for user `de2460c2fdb04a949046acd6a3c0abaa`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:08.499112 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-ca2e3c9a-144a-4435-ad9e-1d461e9bc58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:08.503659 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 800/1590] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:08.504307 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a31c8-f3b8-4ae6-8a7f-6773ab72c0c2 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:08.513447 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a31c8-f3b8-4ae6-8a7f-6773ab72c0c2 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:08.518376 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06de1da-d9c9-4e89-880c-49d5dda0af55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:08.519015 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06de1da-d9c9-4e89-880c-49d5dda0af55 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:08.519296 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06de1da-d9c9-4e89-880c-49d5dda0af55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:08.519565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06de1da-d9c9-4e89-880c-49d5dda0af55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:08.520204 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06de1da-d9c9-4e89-880c-49d5dda0af55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:08.521814 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06de1da-d9c9-4e89-880c-49d5dda0af55 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:08.535180 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 801/1591] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:08.535180 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06de1da-d9c9-4e89-880c-49d5dda0af55 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:08.545639 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06de1da-d9c9-4e89-880c-49d5dda0af55 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:08.551611 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1864ac-6605-411f-8eb4-d84655510a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:08.552204 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1864ac-6605-411f-8eb4-d84655510a02 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:08.552481 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1864ac-6605-411f-8eb4-d84655510a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:08.552751 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1864ac-6605-411f-8eb4-d84655510a02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:08.553383 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1864ac-6605-411f-8eb4-d84655510a02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:08.554979 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1864ac-6605-411f-8eb4-d84655510a02 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:08.565117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 802/1592] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:08.565364 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 791/1593] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:08.569078 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1864ac-6605-411f-8eb4-d84655510a02 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:08.569718 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e3c9a-144a-4435-ad9e-1d461e9bc58d None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:08.578543 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1864ac-6605-411f-8eb4-d84655510a02 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:08.580261 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e3c9a-144a-4435-ad9e-1d461e9bc58d None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:08.583372 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d6f99-cc08-4174-919b-e29bad9c1077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:08.584136 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d6f99-cc08-4174-919b-e29bad9c1077 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:08.584418 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d6f99-cc08-4174-919b-e29bad9c1077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:08.584683 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d6f99-cc08-4174-919b-e29bad9c1077 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:08.585103 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e1627c-48c8-4c7c-a4e5-16f28ec7fdb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:08.585405 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1d6f99-cc08-4174-919b-e29bad9c1077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:08.586277 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e1627c-48c8-4c7c-a4e5-16f28ec7fdb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:08.586277 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e1627c-48c8-4c7c-a4e5-16f28ec7fdb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:08.586277 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e1627c-48c8-4c7c-a4e5-16f28ec7fdb6 admin admin] PATH_INFO: `/v3/projects/701f62e5a7b14dfc86f2dcc5aa39d958/users/0cafcc70e0e74a5fa7c5ce530e0e848f/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:08.587565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1d6f99-cc08-4174-919b-e29bad9c1077 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:08.601712 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 803/1594] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:08.601712 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d6f99-cc08-4174-919b-e29bad9c1077 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:08.604882 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e1627c-48c8-4c7c-a4e5-16f28ec7fdb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=701f62e5a7b14dfc86f2dcc5aa39d958, user_id=0cafcc70e0e74a5fa7c5ce530e0e848f, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:08.607564 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e1627c-48c8-4c7c-a4e5-16f28ec7fdb6 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:08.612606 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d6f99-cc08-4174-919b-e29bad9c1077 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:08.618271 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d8d36-1916-4757-b1a6-26b9730cf5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:08.618926 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d8d36-1916-4757-b1a6-26b9730cf5de admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:08.619280 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d8d36-1916-4757-b1a6-26b9730cf5de admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:08.619550 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d8d36-1916-4757-b1a6-26b9730cf5de admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:08.620183 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996d8d36-1916-4757-b1a6-26b9730cf5de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:08.621746 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996d8d36-1916-4757-b1a6-26b9730cf5de admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:08.622007 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 792/1595] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:13:08 2022] PUT /identity/v3/projects/701f62e5a7b14dfc86f2dcc5aa39d958/users/0cafcc70e0e74a5fa7c5ce530e0e848f/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:08.622822 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e1627c-48c8-4c7c-a4e5-16f28ec7fdb6 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:08.627497 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:08.635417 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 804/1596] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:08.636230 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d8d36-1916-4757-b1a6-26b9730cf5de admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:08.679914 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:08.692009 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d8d36-1916-4757-b1a6-26b9730cf5de admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:08.692620 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e1627c-48c8-4c7c-a4e5-16f28ec7fdb6 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:08.697531 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4fab95-faa3-40d0-9448-1433d32fffb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:08.698148 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4fab95-faa3-40d0-9448-1433d32fffb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:08.698412 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4fab95-faa3-40d0-9448-1433d32fffb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:08.698684 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4fab95-faa3-40d0-9448-1433d32fffb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:08.699357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4fab95-faa3-40d0-9448-1433d32fffb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:08.700947 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4fab95-faa3-40d0-9448-1433d32fffb0 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:08.711385 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 793/1597] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:08.712193 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4fab95-faa3-40d0-9448-1433d32fffb0 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:08.723066 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4fab95-faa3-40d0-9448-1433d32fffb0 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:08.727646 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf613e8-679e-4e40-a80b-399267c36789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:08.728277 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf613e8-679e-4e40-a80b-399267c36789 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:08.728552 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf613e8-679e-4e40-a80b-399267c36789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:08.728820 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf613e8-679e-4e40-a80b-399267c36789 admin admin] PATH_INFO: `/v3/projects/71ab7e50cc3540cb81bdb573293a0329/users/1cd3f9dd67fa44f59c39535cb61272a8/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:08.743549 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf613e8-679e-4e40-a80b-399267c36789 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71ab7e50cc3540cb81bdb573293a0329, user_id=1cd3f9dd67fa44f59c39535cb61272a8, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:08.743806 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38bcb4-e7b3-4a09-a167-3d1d6b08e4e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:08.744439 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38bcb4-e7b3-4a09-a167-3d1d6b08e4e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:08.744743 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38bcb4-e7b3-4a09-a167-3d1d6b08e4e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:08.745165 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38bcb4-e7b3-4a09-a167-3d1d6b08e4e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:08.746221 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf613e8-679e-4e40-a80b-399267c36789 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:08.754635 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc38bcb4-e7b3-4a09-a167-3d1d6b08e4e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:08.756087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc38bcb4-e7b3-4a09-a167-3d1d6b08e4e3 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:08.759306 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 794/1598] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:13:08 2022] PUT /identity/v3/projects/71ab7e50cc3540cb81bdb573293a0329/users/1cd3f9dd67fa44f59c39535cb61272a8/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:08.760106 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf613e8-679e-4e40-a80b-399267c36789 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:08.765926 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 805/1599] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:13:08.772329 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1870325-bf2c-4a50-bbee-456d875f6f5b None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:08.772681 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1870325-bf2c-4a50-bbee-456d875f6f5b None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:08.773023 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1870325-bf2c-4a50-bbee-456d875f6f5b None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:08.779135 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 806/1600] 10.222.0.42 () {54 vars in 962 bytes} [Mon Sep 12 14:13:08 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:13:08.780003 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1870325-bf2c-4a50-bbee-456d875f6f5b None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:08.833667 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf613e8-679e-4e40-a80b-399267c36789 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:08.840953 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c78a13-d4e2-4010-98c0-3deccc48c1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:08.841580 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c78a13-d4e2-4010-98c0-3deccc48c1c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:08.841859 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c78a13-d4e2-4010-98c0-3deccc48c1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:08.842400 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c78a13-d4e2-4010-98c0-3deccc48c1c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:08.843495 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c78a13-d4e2-4010-98c0-3deccc48c1c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:08.845531 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c78a13-d4e2-4010-98c0-3deccc48c1c1 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:08.857173 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1870325-bf2c-4a50-bbee-456d875f6f5b None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:08.857862 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 795/1601] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:08.859208 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572694fd-73d9-4bbe-beb2-8ba4df07d5a1 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:08.859494 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572694fd-73d9-4bbe-beb2-8ba4df07d5a1 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:08.859771 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572694fd-73d9-4bbe-beb2-8ba4df07d5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:08.866686 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f6f91-14d2-4b24-8be7-dbe453052e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:08.867838 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f6f91-14d2-4b24-8be7-dbe453052e68 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:08.868192 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f6f91-14d2-4b24-8be7-dbe453052e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:08.868769 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f6f91-14d2-4b24-8be7-dbe453052e68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:08.869550 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f6f91-14d2-4b24-8be7-dbe453052e68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:08.871376 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f6f91-14d2-4b24-8be7-dbe453052e68 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:08.879629 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-572694fd-73d9-4bbe-beb2-8ba4df07d5a1 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:08.880848 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-572694fd-73d9-4bbe-beb2-8ba4df07d5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:08.882579 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 807/1602] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:08.883513 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f6f91-14d2-4b24-8be7-dbe453052e68 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:08.889722 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 796/1603] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:08.890606 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572694fd-73d9-4bbe-beb2-8ba4df07d5a1 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:08.895230 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f6f91-14d2-4b24-8be7-dbe453052e68 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:08.900740 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f17a96-2211-4cd0-80cb-3d2c20075e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:08.901514 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f17a96-2211-4cd0-80cb-3d2c20075e03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:08.901861 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572694fd-73d9-4bbe-beb2-8ba4df07d5a1 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:08.901984 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f17a96-2211-4cd0-80cb-3d2c20075e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:08.902571 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f17a96-2211-4cd0-80cb-3d2c20075e03 admin admin] PATH_INFO: `/v3/projects/4a72919a47034a2684973f31e4217f3e/users/fff8e93029f447bb8294d9b516eb407f/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:08.908047 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dc984b-3dd6-40bb-83a5-e4ae4a49bd41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:08.909499 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc984b-3dd6-40bb-83a5-e4ae4a49bd41 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:08.909795 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc984b-3dd6-40bb-83a5-e4ae4a49bd41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:08.910108 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc984b-3dd6-40bb-83a5-e4ae4a49bd41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:08.910886 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dc984b-3dd6-40bb-83a5-e4ae4a49bd41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:08.912645 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dc984b-3dd6-40bb-83a5-e4ae4a49bd41 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:08.919951 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f17a96-2211-4cd0-80cb-3d2c20075e03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a72919a47034a2684973f31e4217f3e, user_id=fff8e93029f447bb8294d9b516eb407f, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:08.922647 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f17a96-2211-4cd0-80cb-3d2c20075e03 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:08.923553 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 797/1604] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:08.928023 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb470c5-0107-4bb9-aca9-904634ec9c7c None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:08.928318 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb470c5-0107-4bb9-aca9-904634ec9c7c None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:08.928653 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb470c5-0107-4bb9-aca9-904634ec9c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:08.937763 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 808/1605] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:13:08 2022] PUT /identity/v3/projects/4a72919a47034a2684973f31e4217f3e/users/fff8e93029f447bb8294d9b516eb407f/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:08.939291 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b38ac5-4257-46ec-a660-5d8b41403efd None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:08.939654 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b38ac5-4257-46ec-a660-5d8b41403efd None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:08.940255 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b38ac5-4257-46ec-a660-5d8b41403efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:08.966365 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-6bb470c5-0107-4bb9-aca9-904634ec9c7c None None] MFA Rules not processed for user `1cd3f9dd67fa44f59c39535cb61272a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:08.967569 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-6bb470c5-0107-4bb9-aca9-904634ec9c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:08.987089 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-48b38ac5-4257-46ec-a660-5d8b41403efd None None] MFA Rules not processed for user `854ab2dcec5a48b18f409b33cb61b5da`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:08.988338 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-48b38ac5-4257-46ec-a660-5d8b41403efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:09.020509 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:09.048094 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:09.055861 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 798/1606] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:08 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:09.056829 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb470c5-0107-4bb9-aca9-904634ec9c7c None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:09.068996 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 809/1607] 10.222.0.42 () {58 vars in 1059 bytes} [Mon Sep 12 14:13:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:09.069950 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b38ac5-4257-46ec-a660-5d8b41403efd None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:09.133482 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb470c5-0107-4bb9-aca9-904634ec9c7c None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:09.138638 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8caa5fc-85d5-4b1a-92bc-016e4188805e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:09.139342 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8caa5fc-85d5-4b1a-92bc-016e4188805e admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:09.139706 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8caa5fc-85d5-4b1a-92bc-016e4188805e admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:09.139881 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8caa5fc-85d5-4b1a-92bc-016e4188805e admin admin] PATH_INFO: `/v3/projects/e73505faee9c4bacbf2456e2d69c56f2/users/1c6cd05ddb3d4d14b2a18bc45401e73d/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:09.150405 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b38ac5-4257-46ec-a660-5d8b41403efd None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:09.155788 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e815b5-699a-4261-a218-ea1167c2afd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:09.156466 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e815b5-699a-4261-a218-ea1167c2afd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:09.156756 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e815b5-699a-4261-a218-ea1167c2afd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:09.157018 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e815b5-699a-4261-a218-ea1167c2afd0 admin admin] PATH_INFO: `/v3/projects/366826b14e0e4a93819c18d8d716a254/users/5f2ff5e497b64e49b7ac25a941a06048/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:09.157330 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8caa5fc-85d5-4b1a-92bc-016e4188805e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e73505faee9c4bacbf2456e2d69c56f2, user_id=1c6cd05ddb3d4d14b2a18bc45401e73d, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:09.161027 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8caa5fc-85d5-4b1a-92bc-016e4188805e admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:09.175348 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e815b5-699a-4261-a218-ea1167c2afd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=366826b14e0e4a93819c18d8d716a254, user_id=5f2ff5e497b64e49b7ac25a941a06048, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:09.178208 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e815b5-699a-4261-a218-ea1167c2afd0 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:09.180071 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 799/1608] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:13:09 2022] PUT /identity/v3/projects/e73505faee9c4bacbf2456e2d69c56f2/users/1c6cd05ddb3d4d14b2a18bc45401e73d/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:09.182010 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8caa5fc-85d5-4b1a-92bc-016e4188805e admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:09.185208 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-c8caa5fc-85d5-4b1a-92bc-016e4188805e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:09.198953 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 810/1609] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:13:09 2022] PUT /identity/v3/projects/366826b14e0e4a93819c18d8d716a254/users/5f2ff5e497b64e49b7ac25a941a06048/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:09.199712 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e815b5-699a-4261-a218-ea1167c2afd0 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:09.268461 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8caa5fc-85d5-4b1a-92bc-016e4188805e admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:09.277148 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e815b5-699a-4261-a218-ea1167c2afd0 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:09.282120 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4eb4e2-f68b-48e5-9f64-4f3ac8b13a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:09.283184 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4eb4e2-f68b-48e5-9f64-4f3ac8b13a2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:09.283567 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4eb4e2-f68b-48e5-9f64-4f3ac8b13a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:09.283906 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4eb4e2-f68b-48e5-9f64-4f3ac8b13a2a admin admin] PATH_INFO: `/v3/projects/fed4a4ba0eda499b90fd268cbb2dfa92/users/3dd645b707b34755b0fd9b88ea17540b/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:09.303077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4eb4e2-f68b-48e5-9f64-4f3ac8b13a2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fed4a4ba0eda499b90fd268cbb2dfa92, user_id=3dd645b707b34755b0fd9b88ea17540b, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:09.307063 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4eb4e2-f68b-48e5-9f64-4f3ac8b13a2a admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:09.338093 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319ab64-3f7c-4129-bb7e-05fea6e4195c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:09.338704 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319ab64-3f7c-4129-bb7e-05fea6e4195c service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:09.339010 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319ab64-3f7c-4129-bb7e-05fea6e4195c service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:09.339278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319ab64-3f7c-4129-bb7e-05fea6e4195c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:09.340116 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 811/1610] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:13:09 2022] PUT /identity/v3/projects/fed4a4ba0eda499b90fd268cbb2dfa92/users/3dd645b707b34755b0fd9b88ea17540b/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:09.341860 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d399aa-b85f-4547-86aa-64516992a6c0 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:09.342145 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d399aa-b85f-4547-86aa-64516992a6c0 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:09.342414 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d399aa-b85f-4547-86aa-64516992a6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:09.354804 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6319ab64-3f7c-4129-bb7e-05fea6e4195c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:09.356359 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6319ab64-3f7c-4129-bb7e-05fea6e4195c service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:09.365722 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-84d399aa-b85f-4547-86aa-64516992a6c0 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:09.366962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-84d399aa-b85f-4547-86aa-64516992a6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:09.367905 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 800/1611] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:13:09.368592 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319ab64-3f7c-4129-bb7e-05fea6e4195c service neutron] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:09.382887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319ab64-3f7c-4129-bb7e-05fea6e4195c service neutron] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:09.409161 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:09.423495 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:09.435229 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 812/1612] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:09.435958 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d399aa-b85f-4547-86aa-64516992a6c0 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:09.460406 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5a7b8b-bbdc-4310-8df1-29bcffd2fac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:09.461093 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a7b8b-bbdc-4310-8df1-29bcffd2fac3 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:09.461371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a7b8b-bbdc-4310-8df1-29bcffd2fac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:09.461635 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a7b8b-bbdc-4310-8df1-29bcffd2fac3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:09.462263 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5a7b8b-bbdc-4310-8df1-29bcffd2fac3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:09.464191 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5a7b8b-bbdc-4310-8df1-29bcffd2fac3 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:09.498355 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 801/1613] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:13:09 2022] POST /identity/v3/projects => generated 403 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:09.499784 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39837e-5586-41ef-9a63-6a022b118697 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:09.500246 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39837e-5586-41ef-9a63-6a022b118697 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:09.500526 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39837e-5586-41ef-9a63-6a022b118697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:09.528322 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-0e39837e-5586-41ef-9a63-6a022b118697 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:09.530004 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-0e39837e-5586-41ef-9a63-6a022b118697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:09.540540 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d399aa-b85f-4547-86aa-64516992a6c0 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:09.625179 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 802/1614] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:09.626473 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a63118-06e5-4182-a17f-8407929b3efb None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:09.626754 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a63118-06e5-4182-a17f-8407929b3efb None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:09.627828 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a63118-06e5-4182-a17f-8407929b3efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:09.641192 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d254738e-943c-45d2-8a0c-f62d72b2873c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:09.642975 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254738e-943c-45d2-8a0c-f62d72b2873c service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:09.643201 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254738e-943c-45d2-8a0c-f62d72b2873c service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:09.643641 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254738e-943c-45d2-8a0c-f62d72b2873c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:09.661762 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-89a63118-06e5-4182-a17f-8407929b3efb None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:09.664359 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-89a63118-06e5-4182-a17f-8407929b3efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:09.674884 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 803/1615] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:09.676352 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5599f-7850-4523-97ce-daabc535d5f1 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:09.677231 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5599f-7850-4523-97ce-daabc535d5f1 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:09.677647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5599f-7850-4523-97ce-daabc535d5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:09.718627 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-39c5599f-7850-4523-97ce-daabc535d5f1 None None] MFA Rules not processed for user `3dd645b707b34755b0fd9b88ea17540b`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:09.720209 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-39c5599f-7850-4523-97ce-daabc535d5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:09.748979 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d254738e-943c-45d2-8a0c-f62d72b2873c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:09.750448 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d254738e-943c-45d2-8a0c-f62d72b2873c service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:09.762631 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 813/1616] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:13:09.763676 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d254738e-943c-45d2-8a0c-f62d72b2873c service nova] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:09.811715 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:09.858598 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d254738e-943c-45d2-8a0c-f62d72b2873c service nova] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:09.869885 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621d3ffd-c0b9-4443-ba5d-96aed4810fea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:09.870565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d3ffd-c0b9-4443-ba5d-96aed4810fea service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:09.870864 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d3ffd-c0b9-4443-ba5d-96aed4810fea service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:09.871144 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d3ffd-c0b9-4443-ba5d-96aed4810fea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:09.875404 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 804/1617] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:09 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:09.876324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5599f-7850-4523-97ce-daabc535d5f1 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:09.901147 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621d3ffd-c0b9-4443-ba5d-96aed4810fea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:09.901680 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5599f-7850-4523-97ce-daabc535d5f1 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:09.902485 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621d3ffd-c0b9-4443-ba5d-96aed4810fea service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:09.912360 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd19f9d-2b23-44e8-9302-e48866c8533d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:09.913106 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd19f9d-2b23-44e8-9302-e48866c8533d admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:09.913479 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd19f9d-2b23-44e8-9302-e48866c8533d admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:09.913843 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd19f9d-2b23-44e8-9302-e48866c8533d admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:09.914733 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd19f9d-2b23-44e8-9302-e48866c8533d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:09.917318 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd19f9d-2b23-44e8-9302-e48866c8533d admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:09.935378 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 814/1618] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:13:09.938264 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621d3ffd-c0b9-4443-ba5d-96aed4810fea service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:09.979642 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621d3ffd-c0b9-4443-ba5d-96aed4810fea service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:09.979802 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 805/1619] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:13:09 2022] POST /identity/v3/users => generated 308 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:09.983488 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd19f9d-2b23-44e8-9302-e48866c8533d admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:09.986539 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670dbcf-b898-488c-97c6-3fa88f731d33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:09.988254 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670dbcf-b898-488c-97c6-3fa88f731d33 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:09.989030 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670dbcf-b898-488c-97c6-3fa88f731d33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:09.989515 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670dbcf-b898-488c-97c6-3fa88f731d33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:10.001412 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7670dbcf-b898-488c-97c6-3fa88f731d33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:10.002959 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7670dbcf-b898-488c-97c6-3fa88f731d33 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:10.004322 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd19f9d-2b23-44e8-9302-e48866c8533d admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:10.008122 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e91087a-590c-4aac-a722-e68ab23e67b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:10.008733 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91087a-590c-4aac-a722-e68ab23e67b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:10.009015 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91087a-590c-4aac-a722-e68ab23e67b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:10.009283 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91087a-590c-4aac-a722-e68ab23e67b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:10.015415 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 815/1620] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:13:10.016733 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670dbcf-b898-488c-97c6-3fa88f731d33 service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:10.028175 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e91087a-590c-4aac-a722-e68ab23e67b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:10.029019 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670dbcf-b898-488c-97c6-3fa88f731d33 service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:10.029707 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e91087a-590c-4aac-a722-e68ab23e67b7 service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:10.036738 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e1eea-2a45-495e-a3d5-7744f3f9b1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:10.037796 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e1eea-2a45-495e-a3d5-7744f3f9b1a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:10.038362 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e1eea-2a45-495e-a3d5-7744f3f9b1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:10.038914 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e1eea-2a45-495e-a3d5-7744f3f9b1a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:10.041477 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8e1eea-2a45-495e-a3d5-7744f3f9b1a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:10.043868 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8e1eea-2a45-495e-a3d5-7744f3f9b1a9 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:10.051356 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 806/1621] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:13:10.052207 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e91087a-590c-4aac-a722-e68ab23e67b7 service neutron] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:10.067824 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:10.081598 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e91087a-590c-4aac-a722-e68ab23e67b7 service neutron] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:10.086653 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 816/1622] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:13:10 2022] POST /identity/v3/projects => generated 355 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:10.087883 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e1eea-2a45-495e-a3d5-7744f3f9b1a9 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:10.177348 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab8a06-b966-4408-a50d-f37bc9d53833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:10.179292 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab8a06-b966-4408-a50d-f37bc9d53833 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:10.179584 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab8a06-b966-4408-a50d-f37bc9d53833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:10.179855 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab8a06-b966-4408-a50d-f37bc9d53833 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:10.180577 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ab8a06-b966-4408-a50d-f37bc9d53833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:10.182351 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ab8a06-b966-4408-a50d-f37bc9d53833 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:10.190370 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e1eea-2a45-495e-a3d5-7744f3f9b1a9 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:10.199177 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b788b-8caa-4a78-8c41-b3503697c536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:10.199519 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 807/1623] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:10.200321 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b788b-8caa-4a78-8c41-b3503697c536 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:10.200793 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b788b-8caa-4a78-8c41-b3503697c536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:10.201042 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d02ac-a042-4bda-8b65-58f46dcbf132 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:10.201327 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d02ac-a042-4bda-8b65-58f46dcbf132 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:10.201530 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b788b-8caa-4a78-8c41-b3503697c536 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:10.201652 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d02ac-a042-4bda-8b65-58f46dcbf132 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:10.203177 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 808/1624] 10.222.0.42 () {54 vars in 912 bytes} [Mon Sep 12 14:13:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:13:10.204166 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b788b-8caa-4a78-8c41-b3503697c536 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:10.204434 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d02ac-a042-4bda-8b65-58f46dcbf132 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:10.208177 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b788b-8caa-4a78-8c41-b3503697c536 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:10.227268 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d02ac-a042-4bda-8b65-58f46dcbf132 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:10.237269 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf7dc08-db89-492c-85ed-3f44bc0ceb02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:10.238030 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7dc08-db89-492c-85ed-3f44bc0ceb02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:10.238368 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7dc08-db89-492c-85ed-3f44bc0ceb02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:10.238696 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7dc08-db89-492c-85ed-3f44bc0ceb02 admin admin] PATH_INFO: `/v3/projects/0d6eb9f1296f44c38aa82b3f97b05f44/users/b53913d35de7407abc0ff13719cf2c3a/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:10.241707 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 817/1625] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:13:10 2022] POST /identity/v3/users => generated 284 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:10.243237 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b829399-86c1-4566-b105-8666f1063fbc None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:10.243671 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b829399-86c1-4566-b105-8666f1063fbc None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:10.244092 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b829399-86c1-4566-b105-8666f1063fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:10.264195 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-1b829399-86c1-4566-b105-8666f1063fbc None None] MFA Rules not processed for user `854ab2dcec5a48b18f409b33cb61b5da`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:10.266329 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1b829399-86c1-4566-b105-8666f1063fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:10.272708 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf7dc08-db89-492c-85ed-3f44bc0ceb02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d6eb9f1296f44c38aa82b3f97b05f44, user_id=b53913d35de7407abc0ff13719cf2c3a, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:10.276784 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf7dc08-db89-492c-85ed-3f44bc0ceb02 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:10.302403 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 809/1626] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:13:10 2022] PUT /identity/v3/projects/0d6eb9f1296f44c38aa82b3f97b05f44/users/b53913d35de7407abc0ff13719cf2c3a/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:10.303517 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf7dc08-db89-492c-85ed-3f44bc0ceb02 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:10.371573 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 818/1627] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Sep 12 14:13:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:10.373175 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b829399-86c1-4566-b105-8666f1063fbc None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:10.387224 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b829399-86c1-4566-b105-8666f1063fbc None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:10.406052 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf7dc08-db89-492c-85ed-3f44bc0ceb02 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:10.412532 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a2a12-7530-409b-acc9-d626fbbd12d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:10.413541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a2a12-7530-409b-acc9-d626fbbd12d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:10.413887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a2a12-7530-409b-acc9-d626fbbd12d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:10.414281 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a2a12-7530-409b-acc9-d626fbbd12d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:10.414738 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:10.415134 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a2a12-7530-409b-acc9-d626fbbd12d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:10.416985 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a2a12-7530-409b-acc9-d626fbbd12d8 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:10.430675 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 810/1628] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:10.436430 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a2a12-7530-409b-acc9-d626fbbd12d8 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:10.481559 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0477368b-4e62-4ece-adf5-bab930d02697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:10.482877 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477368b-4e62-4ece-adf5-bab930d02697 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:10.483502 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477368b-4e62-4ece-adf5-bab930d02697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:10.484142 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477368b-4e62-4ece-adf5-bab930d02697 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:10.484946 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0477368b-4e62-4ece-adf5-bab930d02697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:10.487375 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0477368b-4e62-4ece-adf5-bab930d02697 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:10.504117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 819/1629] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:10.505113 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0477368b-4e62-4ece-adf5-bab930d02697 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:10.520237 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0477368b-4e62-4ece-adf5-bab930d02697 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:10.525465 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ef645-5698-47ef-97a6-3ac47bc47944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:10.526243 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ef645-5698-47ef-97a6-3ac47bc47944 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:10.526514 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ef645-5698-47ef-97a6-3ac47bc47944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:10.526791 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ef645-5698-47ef-97a6-3ac47bc47944 admin admin] PATH_INFO: `/v3/projects/8c01e89f44654a9dba00b4858a03fd8f/users/388b52f56dee4ac1ad4007fed6841d0e/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:10.530567 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a2a12-7530-409b-acc9-d626fbbd12d8 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:10.537467 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093dec52-2b84-4c2e-8c55-d4ae4fa99c4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:10.538569 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093dec52-2b84-4c2e-8c55-d4ae4fa99c4d service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:10.539074 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093dec52-2b84-4c2e-8c55-d4ae4fa99c4d service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:10.539840 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093dec52-2b84-4c2e-8c55-d4ae4fa99c4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:10.550123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8ef645-5698-47ef-97a6-3ac47bc47944 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c01e89f44654a9dba00b4858a03fd8f, user_id=388b52f56dee4ac1ad4007fed6841d0e, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:10.553516 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8ef645-5698-47ef-97a6-3ac47bc47944 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:10.573254 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:10.584870 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 820/1630] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:13:10 2022] PUT /identity/v3/projects/8c01e89f44654a9dba00b4858a03fd8f/users/388b52f56dee4ac1ad4007fed6841d0e/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:10.584870 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ef645-5698-47ef-97a6-3ac47bc47944 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:10.660626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093dec52-2b84-4c2e-8c55-d4ae4fa99c4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:10.662992 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093dec52-2b84-4c2e-8c55-d4ae4fa99c4d service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:10.683099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 811/1631] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:13:10.684432 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093dec52-2b84-4c2e-8c55-d4ae4fa99c4d service cinder] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:10.688345 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ef645-5698-47ef-97a6-3ac47bc47944 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:10.695979 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94a87d0-bf64-407f-bfcf-de7addfea282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:10.696924 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a87d0-bf64-407f-bfcf-de7addfea282 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:10.697242 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a87d0-bf64-407f-bfcf-de7addfea282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:10.698470 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a87d0-bf64-407f-bfcf-de7addfea282 admin admin] PATH_INFO: `/v3/projects/0d6eb9f1296f44c38aa82b3f97b05f44/users/b53913d35de7407abc0ff13719cf2c3a/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:10.699080 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093dec52-2b84-4c2e-8c55-d4ae4fa99c4d service cinder] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:10.700675 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94a87d0-bf64-407f-bfcf-de7addfea282 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d6eb9f1296f44c38aa82b3f97b05f44, user_id=b53913d35de7407abc0ff13719cf2c3a, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:10.704360 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94a87d0-bf64-407f-bfcf-de7addfea282 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:10.708921 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e3dcb7-7e51-4a79-8184-538c5a464166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:10.710156 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3dcb7-7e51-4a79-8184-538c5a464166 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:10.710440 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3dcb7-7e51-4a79-8184-538c5a464166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:10.710712 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3dcb7-7e51-4a79-8184-538c5a464166 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:10.711762 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e3dcb7-7e51-4a79-8184-538c5a464166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:10.713779 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e3dcb7-7e51-4a79-8184-538c5a464166 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:10.730743 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 821/1632] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:13:10 2022] PUT /identity/v3/projects/0d6eb9f1296f44c38aa82b3f97b05f44/users/b53913d35de7407abc0ff13719cf2c3a/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:10.742161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0fdb9-4266-4668-a2a1-4a2e98286007 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:10.742582 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0fdb9-4266-4668-a2a1-4a2e98286007 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:10.743520 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0fdb9-4266-4668-a2a1-4a2e98286007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:10.751151 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 812/1633] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:10.756011 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e3dcb7-7e51-4a79-8184-538c5a464166 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:10.792888 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e3dcb7-7e51-4a79-8184-538c5a464166 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:10.802121 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-9cf0fdb9-4266-4668-a2a1-4a2e98286007 None None] MFA Rules not processed for user `b53913d35de7407abc0ff13719cf2c3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:10.803918 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-9cf0fdb9-4266-4668-a2a1-4a2e98286007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:10.924421 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fab835-743b-499e-8d7a-1a448d675cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:10.925662 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fab835-743b-499e-8d7a-1a448d675cea admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:10.926543 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fab835-743b-499e-8d7a-1a448d675cea admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:10.927511 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fab835-743b-499e-8d7a-1a448d675cea admin admin] PATH_INFO: `/v3/projects/8c01e89f44654a9dba00b4858a03fd8f/users/388b52f56dee4ac1ad4007fed6841d0e/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:10.931662 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fab835-743b-499e-8d7a-1a448d675cea admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c01e89f44654a9dba00b4858a03fd8f, user_id=388b52f56dee4ac1ad4007fed6841d0e, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:10.949134 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fab835-743b-499e-8d7a-1a448d675cea admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:10.978789 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 813/1634] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:13:10 2022] PUT /identity/v3/projects/8c01e89f44654a9dba00b4858a03fd8f/users/388b52f56dee4ac1ad4007fed6841d0e/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:13:10.983056 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 822/1635] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:10 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:10.993951 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c1767-f5a7-4d28-acbc-7c063dc62cf9 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:10.994271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c1767-f5a7-4d28-acbc-7c063dc62cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:10.994545 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c1767-f5a7-4d28-acbc-7c063dc62cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:10.996235 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0fdb9-4266-4668-a2a1-4a2e98286007 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:11.092808 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:11.093204 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-769c1767-f5a7-4d28-acbc-7c063dc62cf9 None None] MFA Rules not processed for user `388b52f56dee4ac1ad4007fed6841d0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:11.093204 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-769c1767-f5a7-4d28-acbc-7c063dc62cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:11.140233 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0fdb9-4266-4668-a2a1-4a2e98286007 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:11.211441 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 814/1636] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:10 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:11.218097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf3ced8-c479-479c-8091-dd7f64f7e9b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:11.219046 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3ced8-c479-479c-8091-dd7f64f7e9b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:11.219287 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3ced8-c479-479c-8091-dd7f64f7e9b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:11.219644 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3ced8-c479-479c-8091-dd7f64f7e9b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:11.301755 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf3ced8-c479-479c-8091-dd7f64f7e9b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:11.303549 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf3ced8-c479-479c-8091-dd7f64f7e9b9 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:11.317362 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 823/1637] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:10 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:13:11.403245 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c1767-f5a7-4d28-acbc-7c063dc62cf9 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:11.421725 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c1767-f5a7-4d28-acbc-7c063dc62cf9 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:11.429610 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f0f5b-4fe9-441b-b045-efd431d7340b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:11.430442 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f0f5b-4fe9-441b-b045-efd431d7340b service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:11.430818 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f0f5b-4fe9-441b-b045-efd431d7340b service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:11.431313 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f0f5b-4fe9-441b-b045-efd431d7340b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:11.444051 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8f0f5b-4fe9-441b-b045-efd431d7340b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:11.446370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8f0f5b-4fe9-441b-b045-efd431d7340b service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:11.460332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 815/1638] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:11 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:13:11.703719 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf3ced8-c479-479c-8091-dd7f64f7e9b9 service cinder] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:11.728141 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf3ced8-c479-479c-8091-dd7f64f7e9b9 service cinder] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:11.757116 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c5fe4-ad0c-4c0c-a692-1b0f4e32f8f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:11.758304 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c5fe4-ad0c-4c0c-a692-1b0f4e32f8f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:11.758636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c5fe4-ad0c-4c0c-a692-1b0f4e32f8f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:11.759030 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c5fe4-ad0c-4c0c-a692-1b0f4e32f8f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:11.839345 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f0f5b-4fe9-441b-b045-efd431d7340b service cinder] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:11.854998 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:11.879191 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c5fe4-ad0c-4c0c-a692-1b0f4e32f8f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:11.880634 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c5fe4-ad0c-4c0c-a692-1b0f4e32f8f4 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:11.900999 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 824/1639] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:11 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:13:12.010867 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f0f5b-4fe9-441b-b045-efd431d7340b service cinder] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:12.024113 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa966e4-a547-442e-b988-5a945c9acff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:12.024766 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa966e4-a547-442e-b988-5a945c9acff0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:12.025049 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa966e4-a547-442e-b988-5a945c9acff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:12.025338 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa966e4-a547-442e-b988-5a945c9acff0 admin admin] PATH_INFO: `/v3/users/1cd3f9dd67fa44f59c39535cb61272a8` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:12.034442 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa966e4-a547-442e-b988-5a945c9acff0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cd3f9dd67fa44f59c39535cb61272a8)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:12.036380 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa966e4-a547-442e-b988-5a945c9acff0 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:12.046722 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e842ebc7-52be-4ea2-8906-134ac681d00f None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:12.047393 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e842ebc7-52be-4ea2-8906-134ac681d00f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:12.047624 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e842ebc7-52be-4ea2-8906-134ac681d00f None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:12.049209 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 825/1640] 10.222.0.42 () {54 vars in 962 bytes} [Mon Sep 12 14:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 14:13:12.064480 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d986aa6-7495-4ac6-91f9-75a073032992 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:12.065602 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d986aa6-7495-4ac6-91f9-75a073032992 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:12.066583 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d986aa6-7495-4ac6-91f9-75a073032992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:12.144194 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-8d986aa6-7495-4ac6-91f9-75a073032992 None None] MFA Rules not processed for user `854ab2dcec5a48b18f409b33cb61b5da`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:12.156276 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-8d986aa6-7495-4ac6-91f9-75a073032992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:12.167155 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-efa966e4-a547-442e-b988-5a945c9acff0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cd3f9dd67fa44f59c39535cb61272a8'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:12.189020 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-efa966e4-a547-442e-b988-5a945c9acff0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cd3f9dd67fa44f59c39535cb61272a8'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:12.190365 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-efa966e4-a547-442e-b988-5a945c9acff0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cd3f9dd67fa44f59c39535cb61272a8'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:12.191050 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-efa966e4-a547-442e-b988-5a945c9acff0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cd3f9dd67fa44f59c39535cb61272a8'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:12.206284 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-efa966e4-a547-442e-b988-5a945c9acff0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cd3f9dd67fa44f59c39535cb61272a8'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:12.218779 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:12.243312 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 816/1641] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:13:11 2022] DELETE /identity/v3/users/1cd3f9dd67fa44f59c39535cb61272a8 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:12.249329 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd73c7-4510-46d6-bf96-324bc2dbdfb6 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:12.249329 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd73c7-4510-46d6-bf96-324bc2dbdfb6 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:12.249329 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd73c7-4510-46d6-bf96-324bc2dbdfb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:12.306333 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-eafd73c7-4510-46d6-bf96-324bc2dbdfb6 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:12.310730 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-eafd73c7-4510-46d6-bf96-324bc2dbdfb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:12.402258 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 826/1642] 10.222.0.42 () {58 vars in 1059 bytes} [Mon Sep 12 14:13:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:12.407521 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc3d6b-bcf9-41c2-9f1b-efa2cb2593d0 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:12.408043 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc3d6b-bcf9-41c2-9f1b-efa2cb2593d0 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:12.408540 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc3d6b-bcf9-41c2-9f1b-efa2cb2593d0 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:12.419016 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 827/1643] 10.222.0.42 () {54 vars in 912 bytes} [Mon Sep 12 14:13:12 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:13:12.421358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a045ac-af86-4855-8ecc-43c5033e8b91 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:12.425326 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a045ac-af86-4855-8ecc-43c5033e8b91 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:12.425783 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a045ac-af86-4855-8ecc-43c5033e8b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:12.459794 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-29a045ac-af86-4855-8ecc-43c5033e8b91 None None] MFA Rules not processed for user `854ab2dcec5a48b18f409b33cb61b5da`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:12.461107 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-29a045ac-af86-4855-8ecc-43c5033e8b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:12.501406 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 817/1644] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:12.514050 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd73c7-4510-46d6-bf96-324bc2dbdfb6 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:12.516418 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-eafd73c7-4510-46d6-bf96-324bc2dbdfb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:12.559237 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:12.608756 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd73c7-4510-46d6-bf96-324bc2dbdfb6 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:12.613072 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 828/1645] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Sep 12 14:13:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:12.628592 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a045ac-af86-4855-8ecc-43c5033e8b91 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:12.630945 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690183d5-e2a2-4767-9bc0-bd95d6de29a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:12.639751 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690183d5-e2a2-4767-9bc0-bd95d6de29a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:12.640177 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690183d5-e2a2-4767-9bc0-bd95d6de29a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:12.640816 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-29a045ac-af86-4855-8ecc-43c5033e8b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:12.641205 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690183d5-e2a2-4767-9bc0-bd95d6de29a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:12.662993 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690183d5-e2a2-4767-9bc0-bd95d6de29a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:12.664604 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690183d5-e2a2-4767-9bc0-bd95d6de29a9 service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:12.683210 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 818/1646] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:13:12.725187 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a045ac-af86-4855-8ecc-43c5033e8b91 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:12.729829 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700e260-8088-4659-bed6-58fa5eafa111 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:12.730574 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700e260-8088-4659-bed6-58fa5eafa111 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:12.731074 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700e260-8088-4659-bed6-58fa5eafa111 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:12.731575 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700e260-8088-4659-bed6-58fa5eafa111 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:12.743605 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5700e260-8088-4659-bed6-58fa5eafa111 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:12.745182 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5700e260-8088-4659-bed6-58fa5eafa111 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:12.767445 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 829/1647] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:13:14.207010 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690183d5-e2a2-4767-9bc0-bd95d6de29a9 service neutron] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:14.208527 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-690183d5-e2a2-4767-9bc0-bd95d6de29a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:14.230791 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690183d5-e2a2-4767-9bc0-bd95d6de29a9 service neutron] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:14.240642 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fbc431-563a-4b26-a225-f496bb0423b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:14.241294 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fbc431-563a-4b26-a225-f496bb0423b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:14.241656 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fbc431-563a-4b26-a225-f496bb0423b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:14.241851 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fbc431-563a-4b26-a225-f496bb0423b4 admin admin] PATH_INFO: `/v3/users/de2460c2fdb04a949046acd6a3c0abaa` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:14.243026 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fbc431-563a-4b26-a225-f496bb0423b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de2460c2fdb04a949046acd6a3c0abaa)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:14.244943 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fbc431-563a-4b26-a225-f496bb0423b4 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:14.286806 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700e260-8088-4659-bed6-58fa5eafa111 service cinder] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:14.289179 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-5700e260-8088-4659-bed6-58fa5eafa111 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:14.326003 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-11fbc431-563a-4b26-a225-f496bb0423b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de2460c2fdb04a949046acd6a3c0abaa'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:14.328335 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700e260-8088-4659-bed6-58fa5eafa111 service cinder] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:14.343818 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-11fbc431-563a-4b26-a225-f496bb0423b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de2460c2fdb04a949046acd6a3c0abaa'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:14.344793 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-11fbc431-563a-4b26-a225-f496bb0423b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de2460c2fdb04a949046acd6a3c0abaa'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:14.345432 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-11fbc431-563a-4b26-a225-f496bb0423b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de2460c2fdb04a949046acd6a3c0abaa'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:14.346170 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c234ebd-ed9b-4d12-98b8-54491b93f09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:14.351227 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c234ebd-ed9b-4d12-98b8-54491b93f09d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:14.351769 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c234ebd-ed9b-4d12-98b8-54491b93f09d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:14.352269 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c234ebd-ed9b-4d12-98b8-54491b93f09d admin admin] PATH_INFO: `/v3/projects/71ab7e50cc3540cb81bdb573293a0329` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:14.355951 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c234ebd-ed9b-4d12-98b8-54491b93f09d admin admin] RBAC: Authorizing `identity:delete_project(project_id=71ab7e50cc3540cb81bdb573293a0329)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:14.356082 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-11fbc431-563a-4b26-a225-f496bb0423b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de2460c2fdb04a949046acd6a3c0abaa'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:14.359779 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c234ebd-ed9b-4d12-98b8-54491b93f09d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:14.396319 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 819/1648] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:13:14 2022] DELETE /identity/v3/users/de2460c2fdb04a949046acd6a3c0abaa => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:14.400907 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26463f43-db16-4d68-944b-50f81ccda1f9 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:14.401194 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26463f43-db16-4d68-944b-50f81ccda1f9 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:14.401472 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26463f43-db16-4d68-944b-50f81ccda1f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:14.436806 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:14.449620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-3c234ebd-ed9b-4d12-98b8-54491b93f09d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71ab7e50cc3540cb81bdb573293a0329', 'request_id': 'req-3c234ebd-ed9b-4d12-98b8-54491b93f09d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:14.464486 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-3c234ebd-ed9b-4d12-98b8-54491b93f09d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71ab7e50cc3540cb81bdb573293a0329', 'request_id': 'req-3c234ebd-ed9b-4d12-98b8-54491b93f09d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:14.472369 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-26463f43-db16-4d68-944b-50f81ccda1f9 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:14.475625 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-26463f43-db16-4d68-944b-50f81ccda1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:14.480319 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-3c234ebd-ed9b-4d12-98b8-54491b93f09d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71ab7e50cc3540cb81bdb573293a0329', 'request_id': 'req-3c234ebd-ed9b-4d12-98b8-54491b93f09d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:14.506895 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-3c234ebd-ed9b-4d12-98b8-54491b93f09d admin admin] The token cache is being invalidate because project 71ab7e50cc3540cb81bdb573293a0329 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:13:14.507152 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-3c234ebd-ed9b-4d12-98b8-54491b93f09d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:14.508453 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-3c234ebd-ed9b-4d12-98b8-54491b93f09d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:14.515801 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 830/1649] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:13:14 2022] DELETE /identity/v3/projects/71ab7e50cc3540cb81bdb573293a0329 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:13:14.521502 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c234ebd-ed9b-4d12-98b8-54491b93f09d admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:14.524826 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-3c234ebd-ed9b-4d12-98b8-54491b93f09d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:14.646355 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 820/1650] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:14.654690 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c234ebd-ed9b-4d12-98b8-54491b93f09d admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:14.670098 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26463f43-db16-4d68-944b-50f81ccda1f9 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:14.671750 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-26463f43-db16-4d68-944b-50f81ccda1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:14.679881 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e71f28f-4a66-4baa-aec7-1d41be3d6c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:14.680551 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71f28f-4a66-4baa-aec7-1d41be3d6c95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:14.680833 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71f28f-4a66-4baa-aec7-1d41be3d6c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:14.691270 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71f28f-4a66-4baa-aec7-1d41be3d6c95 admin admin] PATH_INFO: `/v3/users/b53913d35de7407abc0ff13719cf2c3a` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:14.691270 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e71f28f-4a66-4baa-aec7-1d41be3d6c95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b53913d35de7407abc0ff13719cf2c3a)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:14.694511 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: WARNING py.warnings [None req-1e71f28f-4a66-4baa-aec7-1d41be3d6c95 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 14:13:14.694511 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: warnings.warn(msg) Sep 12 14:13:14.694511 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: Sep 12 14:13:14.695786 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e71f28f-4a66-4baa-aec7-1d41be3d6c95 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:14.759636 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:14.795504 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26463f43-db16-4d68-944b-50f81ccda1f9 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:14.801799 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7933379-397c-4da6-ae9a-f403a1877320 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:14.803038 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7933379-397c-4da6-ae9a-f403a1877320 service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:14.803513 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7933379-397c-4da6-ae9a-f403a1877320 service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:14.804065 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7933379-397c-4da6-ae9a-f403a1877320 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:14.827141 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7933379-397c-4da6-ae9a-f403a1877320 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:14.829033 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7933379-397c-4da6-ae9a-f403a1877320 service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:14.850069 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 821/1651] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:13:14.868190 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-1e71f28f-4a66-4baa-aec7-1d41be3d6c95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b53913d35de7407abc0ff13719cf2c3a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:14.889366 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-1e71f28f-4a66-4baa-aec7-1d41be3d6c95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b53913d35de7407abc0ff13719cf2c3a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:14.891164 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-1e71f28f-4a66-4baa-aec7-1d41be3d6c95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b53913d35de7407abc0ff13719cf2c3a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:14.892045 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-1e71f28f-4a66-4baa-aec7-1d41be3d6c95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b53913d35de7407abc0ff13719cf2c3a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:14.932754 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-1e71f28f-4a66-4baa-aec7-1d41be3d6c95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b53913d35de7407abc0ff13719cf2c3a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:15.001383 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 831/1652] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:13:14 2022] DELETE /identity/v3/users/b53913d35de7407abc0ff13719cf2c3a => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:15.581617 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7933379-397c-4da6-ae9a-f403a1877320 service neutron] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:15.594994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7933379-397c-4da6-ae9a-f403a1877320 service neutron] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:15.771789 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada3a93-8cca-46e0-8fed-5a8c42fecc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:15.772528 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada3a93-8cca-46e0-8fed-5a8c42fecc45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:15.772876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada3a93-8cca-46e0-8fed-5a8c42fecc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:15.778257 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada3a93-8cca-46e0-8fed-5a8c42fecc45 admin admin] PATH_INFO: `/v3/projects/0d6eb9f1296f44c38aa82b3f97b05f44` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:15.779788 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada3a93-8cca-46e0-8fed-5a8c42fecc45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d6eb9f1296f44c38aa82b3f97b05f44)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:15.781274 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: WARNING py.warnings [None req-eada3a93-8cca-46e0-8fed-5a8c42fecc45 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 14:13:15.781274 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: warnings.warn(msg) Sep 12 14:13:15.781274 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: Sep 12 14:13:15.784261 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada3a93-8cca-46e0-8fed-5a8c42fecc45 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:15.931708 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-eada3a93-8cca-46e0-8fed-5a8c42fecc45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d6eb9f1296f44c38aa82b3f97b05f44', 'request_id': 'req-eada3a93-8cca-46e0-8fed-5a8c42fecc45'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:15.939942 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-eada3a93-8cca-46e0-8fed-5a8c42fecc45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d6eb9f1296f44c38aa82b3f97b05f44', 'request_id': 'req-eada3a93-8cca-46e0-8fed-5a8c42fecc45'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:15.954371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-eada3a93-8cca-46e0-8fed-5a8c42fecc45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d6eb9f1296f44c38aa82b3f97b05f44', 'request_id': 'req-eada3a93-8cca-46e0-8fed-5a8c42fecc45'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:16.000964 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-eada3a93-8cca-46e0-8fed-5a8c42fecc45 admin admin] The token cache is being invalidate because project 0d6eb9f1296f44c38aa82b3f97b05f44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:13:16.004778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-eada3a93-8cca-46e0-8fed-5a8c42fecc45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:16.006135 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-eada3a93-8cca-46e0-8fed-5a8c42fecc45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:16.012756 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 822/1653] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:13:15 2022] DELETE /identity/v3/projects/0d6eb9f1296f44c38aa82b3f97b05f44 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:16.014392 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d5f50-74d2-44b4-93e8-0210f6786d49 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:16.014942 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d5f50-74d2-44b4-93e8-0210f6786d49 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:16.015041 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d5f50-74d2-44b4-93e8-0210f6786d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:16.046909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-813d5f50-74d2-44b4-93e8-0210f6786d49 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:16.048644 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-813d5f50-74d2-44b4-93e8-0210f6786d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:16.128895 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:16.164929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 832/1654] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:16.166711 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac8ba3-5aa9-462f-9ab6-dccb02186703 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:16.167081 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac8ba3-5aa9-462f-9ab6-dccb02186703 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:16.167379 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac8ba3-5aa9-462f-9ab6-dccb02186703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:16.256253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-4dac8ba3-5aa9-462f-9ab6-dccb02186703 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:16.257567 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-4dac8ba3-5aa9-462f-9ab6-dccb02186703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:16.271459 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 823/1655] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:16.272635 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d5f50-74d2-44b4-93e8-0210f6786d49 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:16.291005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d5f50-74d2-44b4-93e8-0210f6786d49 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:16.308093 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4193af6-3879-421a-8142-684293e6af52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:16.308701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4193af6-3879-421a-8142-684293e6af52 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:16.308831 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4193af6-3879-421a-8142-684293e6af52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:16.309053 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4193af6-3879-421a-8142-684293e6af52 admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:16.310196 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4193af6-3879-421a-8142-684293e6af52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:13:16.310512 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4193af6-3879-421a-8142-684293e6af52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:16.312166 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4193af6-3879-421a-8142-684293e6af52 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:16.344108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 833/1656] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:13:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:13:16.349230 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac8ba3-5aa9-462f-9ab6-dccb02186703 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:16.362786 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac8ba3-5aa9-462f-9ab6-dccb02186703 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:16.380090 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ce930-3b0e-4e27-9698-3fde2fd86ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:16.380742 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ce930-3b0e-4e27-9698-3fde2fd86ffa admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:16.381022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ce930-3b0e-4e27-9698-3fde2fd86ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:16.381294 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ce930-3b0e-4e27-9698-3fde2fd86ffa admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:16.382409 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ce930-3b0e-4e27-9698-3fde2fd86ffa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:16.384391 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ce930-3b0e-4e27-9698-3fde2fd86ffa admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:16.460894 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 824/1657] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:13:16 2022] POST /identity/v3/projects => generated 365 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:16.465093 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4193af6-3879-421a-8142-684293e6af52 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:16.480384 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4193af6-3879-421a-8142-684293e6af52 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:16.597964 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9c78c6-f7cf-4bbc-a086-de03afdbe92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:16.598714 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c78c6-f7cf-4bbc-a086-de03afdbe92d admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:16.599045 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c78c6-f7cf-4bbc-a086-de03afdbe92d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:16.599321 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c78c6-f7cf-4bbc-a086-de03afdbe92d admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:16.599971 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9c78c6-f7cf-4bbc-a086-de03afdbe92d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:16.605967 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9c78c6-f7cf-4bbc-a086-de03afdbe92d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:16.669894 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 834/1658] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:13:16 2022] POST /identity/v3/users => generated 283 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:16.674107 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ce930-3b0e-4e27-9698-3fde2fd86ffa admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:16.703414 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ce930-3b0e-4e27-9698-3fde2fd86ffa admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:16.718354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a706fb2f-6f0d-43b2-9684-6dddbbda6879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:16.719249 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706fb2f-6f0d-43b2-9684-6dddbbda6879 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:16.719838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706fb2f-6f0d-43b2-9684-6dddbbda6879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:16.720245 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706fb2f-6f0d-43b2-9684-6dddbbda6879 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:16.721095 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a706fb2f-6f0d-43b2-9684-6dddbbda6879 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:16.725103 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a706fb2f-6f0d-43b2-9684-6dddbbda6879 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:16.749135 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 825/1659] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:16.753469 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9c78c6-f7cf-4bbc-a086-de03afdbe92d admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:16.781815 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9c78c6-f7cf-4bbc-a086-de03afdbe92d admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:16.792620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c243ec-c1ca-44b8-96b1-22846e399dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:16.794193 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c243ec-c1ca-44b8-96b1-22846e399dc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:16.794489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c243ec-c1ca-44b8-96b1-22846e399dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:16.795598 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c243ec-c1ca-44b8-96b1-22846e399dc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:16.796600 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c243ec-c1ca-44b8-96b1-22846e399dc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:16.798525 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c243ec-c1ca-44b8-96b1-22846e399dc8 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:16.839451 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 835/1660] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:16.843695 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a706fb2f-6f0d-43b2-9684-6dddbbda6879 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:16.861020 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a706fb2f-6f0d-43b2-9684-6dddbbda6879 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:16.868819 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f5e02c-0ab6-43f2-92da-cde68e8419f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:16.869658 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f5e02c-0ab6-43f2-92da-cde68e8419f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:16.869938 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f5e02c-0ab6-43f2-92da-cde68e8419f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:16.870329 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f5e02c-0ab6-43f2-92da-cde68e8419f2 admin admin] PATH_INFO: `/v3/projects/c20b4e9d9c6e46bbbbeb56e7ac472859/users/287bcbae81f34d5687dea75ff9569eda/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:16.922445 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f5e02c-0ab6-43f2-92da-cde68e8419f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c20b4e9d9c6e46bbbbeb56e7ac472859, user_id=287bcbae81f34d5687dea75ff9569eda, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:16.925287 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f5e02c-0ab6-43f2-92da-cde68e8419f2 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:16.962178 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 826/1661] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:13:16 2022] PUT /identity/v3/projects/c20b4e9d9c6e46bbbbeb56e7ac472859/users/287bcbae81f34d5687dea75ff9569eda/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:16.962923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487174e-ec1c-4e20-bc41-e19289932eb2 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:16.963387 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487174e-ec1c-4e20-bc41-e19289932eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:16.963796 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487174e-ec1c-4e20-bc41-e19289932eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:17.069179 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-6487174e-ec1c-4e20-bc41-e19289932eb2 None None] MFA Rules not processed for user `287bcbae81f34d5687dea75ff9569eda`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:17.072859 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-6487174e-ec1c-4e20-bc41-e19289932eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:17.229664 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f5e02c-0ab6-43f2-92da-cde68e8419f2 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:17.250550 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-12f5e02c-0ab6-43f2-92da-cde68e8419f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:17.314139 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:17.329215 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 836/1662] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:17.336712 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487174e-ec1c-4e20-bc41-e19289932eb2 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:17.489271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f5e02c-0ab6-43f2-92da-cde68e8419f2 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:17.503648 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93460c59-7e41-48fc-865b-d088397dfbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:17.504722 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93460c59-7e41-48fc-865b-d088397dfbd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:17.504850 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93460c59-7e41-48fc-865b-d088397dfbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:17.506477 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93460c59-7e41-48fc-865b-d088397dfbd2 admin admin] PATH_INFO: `/v3/projects/cc1e89bb0f744de5b1c520d488ae1b25` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:17.508275 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93460c59-7e41-48fc-865b-d088397dfbd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc1e89bb0f744de5b1c520d488ae1b25)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:17.508649 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:17.510436 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93460c59-7e41-48fc-865b-d088397dfbd2 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:17.578131 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487174e-ec1c-4e20-bc41-e19289932eb2 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:17.584663 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26611beb-0277-4445-ba89-d32d780b2e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:17.599591 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26611beb-0277-4445-ba89-d32d780b2e2d admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:17.599981 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26611beb-0277-4445-ba89-d32d780b2e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:17.600261 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26611beb-0277-4445-ba89-d32d780b2e2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:17.601130 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26611beb-0277-4445-ba89-d32d780b2e2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:17.603384 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26611beb-0277-4445-ba89-d32d780b2e2d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:17.657688 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-93460c59-7e41-48fc-865b-d088397dfbd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc1e89bb0f744de5b1c520d488ae1b25', 'request_id': 'req-93460c59-7e41-48fc-865b-d088397dfbd2'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:17.668127 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-93460c59-7e41-48fc-865b-d088397dfbd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc1e89bb0f744de5b1c520d488ae1b25', 'request_id': 'req-93460c59-7e41-48fc-865b-d088397dfbd2'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:17.674975 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 837/1663] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:13:17 2022] POST /identity/v3/projects => generated 363 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:13:17.680067 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26611beb-0277-4445-ba89-d32d780b2e2d admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:17.683417 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-93460c59-7e41-48fc-865b-d088397dfbd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc1e89bb0f744de5b1c520d488ae1b25', 'request_id': 'req-93460c59-7e41-48fc-865b-d088397dfbd2'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:17.727755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-93460c59-7e41-48fc-865b-d088397dfbd2 admin admin] The token cache is being invalidate because project cc1e89bb0f744de5b1c520d488ae1b25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:13:17.728062 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-93460c59-7e41-48fc-865b-d088397dfbd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:17.728887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-93460c59-7e41-48fc-865b-d088397dfbd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:17.735489 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 827/1664] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:13:17 2022] DELETE /identity/v3/projects/cc1e89bb0f744de5b1c520d488ae1b25 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:17.740078 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292ce0e-6b24-4d3a-bd3e-b51da7cf912c None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:17.740908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292ce0e-6b24-4d3a-bd3e-b51da7cf912c None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:17.741201 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292ce0e-6b24-4d3a-bd3e-b51da7cf912c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:17.812446 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-6292ce0e-6b24-4d3a-bd3e-b51da7cf912c None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:17.813935 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-6292ce0e-6b24-4d3a-bd3e-b51da7cf912c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:17.907339 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26611beb-0277-4445-ba89-d32d780b2e2d admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:17.936885 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ad68c-8e54-40b0-b0a0-08f9b4dc1e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:17.937806 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ad68c-8e54-40b0-b0a0-08f9b4dc1e09 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:17.938093 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ad68c-8e54-40b0-b0a0-08f9b4dc1e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:17.938679 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ad68c-8e54-40b0-b0a0-08f9b4dc1e09 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:17.940541 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9ad68c-8e54-40b0-b0a0-08f9b4dc1e09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:17.947847 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9ad68c-8e54-40b0-b0a0-08f9b4dc1e09 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:17.960725 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 828/1665] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:17.965454 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8eea95-0bea-46d3-86bb-3ea89b24453e None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:17.965751 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8eea95-0bea-46d3-86bb-3ea89b24453e None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:17.966026 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8eea95-0bea-46d3-86bb-3ea89b24453e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:18.016417 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 838/1666] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:13:17 2022] POST /identity/v3/users => generated 288 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:18.022512 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ad68c-8e54-40b0-b0a0-08f9b4dc1e09 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:18.025158 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-8d8eea95-0bea-46d3-86bb-3ea89b24453e None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:18.026739 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-8d8eea95-0bea-46d3-86bb-3ea89b24453e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:18.027610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-6f9ad68c-8e54-40b0-b0a0-08f9b4dc1e09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:18.058359 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 829/1667] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:18.059645 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ad68c-8e54-40b0-b0a0-08f9b4dc1e09 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:18.060884 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8eea95-0bea-46d3-86bb-3ea89b24453e None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:18.074795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3a165-0796-497a-92ad-2760a632b9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:18.079667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3a165-0796-497a-92ad-2760a632b9ae admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:18.079667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3a165-0796-497a-92ad-2760a632b9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:18.079667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3a165-0796-497a-92ad-2760a632b9ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:18.079667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b3a165-0796-497a-92ad-2760a632b9ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:18.080522 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8eea95-0bea-46d3-86bb-3ea89b24453e None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:18.083959 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b3a165-0796-497a-92ad-2760a632b9ae admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:18.101311 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90495229-87a4-4f7d-b09e-c22a4750d571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:18.101873 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90495229-87a4-4f7d-b09e-c22a4750d571 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:18.102150 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90495229-87a4-4f7d-b09e-c22a4750d571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:18.106113 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90495229-87a4-4f7d-b09e-c22a4750d571 admin admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:18.109280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90495229-87a4-4f7d-b09e-c22a4750d571 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:13:18.110077 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90495229-87a4-4f7d-b09e-c22a4750d571 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:18.115270 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90495229-87a4-4f7d-b09e-c22a4750d571 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:18.124864 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 839/1668] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:18 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:18.131996 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3a165-0796-497a-92ad-2760a632b9ae admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:18.169297 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 830/1669] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:13:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:13:18.172233 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3a165-0796-497a-92ad-2760a632b9ae admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:18.174516 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90495229-87a4-4f7d-b09e-c22a4750d571 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:18.185135 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96362e82-8fff-44b4-a215-26fcb852d775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:18.185824 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96362e82-8fff-44b4-a215-26fcb852d775 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:18.186105 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96362e82-8fff-44b4-a215-26fcb852d775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:18.186379 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96362e82-8fff-44b4-a215-26fcb852d775 admin admin] PATH_INFO: `/v3/projects/1aa08aeb75f94056b5530257d1396e91/users/ada88975e82445c596ea4eca15b7794d/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:18.194536 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90495229-87a4-4f7d-b09e-c22a4750d571 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:18.215872 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d0e02-696d-40bb-b2b7-24a43f25b4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:18.219552 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d0e02-696d-40bb-b2b7-24a43f25b4ea admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:18.219877 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d0e02-696d-40bb-b2b7-24a43f25b4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:18.220153 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d0e02-696d-40bb-b2b7-24a43f25b4ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:18.222604 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d0e02-696d-40bb-b2b7-24a43f25b4ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:18.234599 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d0e02-696d-40bb-b2b7-24a43f25b4ea admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:18.245901 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96362e82-8fff-44b4-a215-26fcb852d775 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aa08aeb75f94056b5530257d1396e91, user_id=ada88975e82445c596ea4eca15b7794d, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:18.255515 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96362e82-8fff-44b4-a215-26fcb852d775 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:18.286933 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 840/1670] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:13:18 2022] PUT /identity/v3/projects/1aa08aeb75f94056b5530257d1396e91/users/ada88975e82445c596ea4eca15b7794d/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:13:18.291650 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96362e82-8fff-44b4-a215-26fcb852d775 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:18.306745 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:18.331901 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 831/1671] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:13:18 2022] POST /identity/v3/projects => generated 367 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:18.332857 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d0e02-696d-40bb-b2b7-24a43f25b4ea admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:18.490694 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96362e82-8fff-44b4-a215-26fcb852d775 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:18.554479 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d0e02-696d-40bb-b2b7-24a43f25b4ea admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:18.577233 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d13a21-49fb-4eb7-9008-447b16a63233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:18.577881 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d13a21-49fb-4eb7-9008-447b16a63233 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:18.578167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d13a21-49fb-4eb7-9008-447b16a63233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:18.578443 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d13a21-49fb-4eb7-9008-447b16a63233 admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:18.583321 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d13a21-49fb-4eb7-9008-447b16a63233 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:18.585287 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d13a21-49fb-4eb7-9008-447b16a63233 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:18.609247 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:18.668883 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 832/1672] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:13:18 2022] POST /identity/v3/users => generated 284 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:18.672168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c531a-562a-40ae-b039-5aa3cdf2574c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:18.672963 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c531a-562a-40ae-b039-5aa3cdf2574c admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:18.673088 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d13a21-49fb-4eb7-9008-447b16a63233 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:18.673252 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c531a-562a-40ae-b039-5aa3cdf2574c admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:18.673543 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c531a-562a-40ae-b039-5aa3cdf2574c admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:18.675645 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20c531a-562a-40ae-b039-5aa3cdf2574c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:18.678256 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20c531a-562a-40ae-b039-5aa3cdf2574c admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:18.697100 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 841/1673] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:18 2022] GET /identity/v3/roles => generated 1420 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:18.698229 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c531a-562a-40ae-b039-5aa3cdf2574c admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:18.703873 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d13a21-49fb-4eb7-9008-447b16a63233 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:18.726471 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c531a-562a-40ae-b039-5aa3cdf2574c admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:18.741679 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b15ba-5285-45e0-9d5c-aa81d98b39f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:18.743406 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b15ba-5285-45e0-9d5c-aa81d98b39f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:18.743693 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b15ba-5285-45e0-9d5c-aa81d98b39f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:18.745239 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b15ba-5285-45e0-9d5c-aa81d98b39f2 admin admin] PATH_INFO: `/v3/projects/1aa08aeb75f94056b5530257d1396e91/users/ada88975e82445c596ea4eca15b7794d/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:18.747055 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa99cb-9d5f-41bc-807a-76440921e14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:18.747824 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa99cb-9d5f-41bc-807a-76440921e14a admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:18.748110 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa99cb-9d5f-41bc-807a-76440921e14a admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:18.748378 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa99cb-9d5f-41bc-807a-76440921e14a admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:18.749033 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa99cb-9d5f-41bc-807a-76440921e14a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:18.750636 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa99cb-9d5f-41bc-807a-76440921e14a admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:18.755352 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95b15ba-5285-45e0-9d5c-aa81d98b39f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aa08aeb75f94056b5530257d1396e91, user_id=ada88975e82445c596ea4eca15b7794d, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:18.759174 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95b15ba-5285-45e0-9d5c-aa81d98b39f2 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:18.771522 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 833/1674] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:18 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:18.780025 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa99cb-9d5f-41bc-807a-76440921e14a admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:18.801851 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa99cb-9d5f-41bc-807a-76440921e14a admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:18.802480 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 842/1675] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:13:18 2022] PUT /identity/v3/projects/1aa08aeb75f94056b5530257d1396e91/users/ada88975e82445c596ea4eca15b7794d/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:18.806312 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a424d45-1143-40a3-911c-0db9af7fa1af None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:18.806628 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a424d45-1143-40a3-911c-0db9af7fa1af None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:18.806934 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a424d45-1143-40a3-911c-0db9af7fa1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:18.891405 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-6a424d45-1143-40a3-911c-0db9af7fa1af None None] MFA Rules not processed for user `ada88975e82445c596ea4eca15b7794d`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:18.893527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-6a424d45-1143-40a3-911c-0db9af7fa1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:18.916230 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec106ef6-74b4-40d7-af58-7b57a1691865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:18.917057 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec106ef6-74b4-40d7-af58-7b57a1691865 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:18.917463 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec106ef6-74b4-40d7-af58-7b57a1691865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:18.917739 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec106ef6-74b4-40d7-af58-7b57a1691865 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:18.918451 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec106ef6-74b4-40d7-af58-7b57a1691865 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:18.920884 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec106ef6-74b4-40d7-af58-7b57a1691865 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:18.935873 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 834/1676] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:18 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:18.940066 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec106ef6-74b4-40d7-af58-7b57a1691865 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:18.953032 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec106ef6-74b4-40d7-af58-7b57a1691865 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:18.959577 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8adaa16-896a-4975-8463-08113e7c7ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:18.961034 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8adaa16-896a-4975-8463-08113e7c7ab6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:18.962071 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8adaa16-896a-4975-8463-08113e7c7ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:18.963979 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8adaa16-896a-4975-8463-08113e7c7ab6 admin admin] PATH_INFO: `/v3/projects/f7958feaecbe4664826f14b8717e0525/users/6c5b60e014114e72afd135da80c158ae/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:19.006151 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8adaa16-896a-4975-8463-08113e7c7ab6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7958feaecbe4664826f14b8717e0525, user_id=6c5b60e014114e72afd135da80c158ae, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:19.010391 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8adaa16-896a-4975-8463-08113e7c7ab6 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:19.050858 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 835/1677] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:13:18 2022] PUT /identity/v3/projects/f7958feaecbe4664826f14b8717e0525/users/6c5b60e014114e72afd135da80c158ae/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:13:19.071961 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2432f05-41bd-4d62-a7bd-ba18864dff50 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:19.072344 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 843/1678] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:18 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:19.072942 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2432f05-41bd-4d62-a7bd-ba18864dff50 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:19.074433 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2432f05-41bd-4d62-a7bd-ba18864dff50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:19.079498 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a424d45-1143-40a3-911c-0db9af7fa1af None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:19.082633 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-6a424d45-1143-40a3-911c-0db9af7fa1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:19.154773 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-d2432f05-41bd-4d62-a7bd-ba18864dff50 None None] MFA Rules not processed for user `6c5b60e014114e72afd135da80c158ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:19.158528 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-d2432f05-41bd-4d62-a7bd-ba18864dff50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:19.231200 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a424d45-1143-40a3-911c-0db9af7fa1af None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:19.289006 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:19.346658 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 836/1679] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:13:19.357109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2432f05-41bd-4d62-a7bd-ba18864dff50 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:19.360007 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-d2432f05-41bd-4d62-a7bd-ba18864dff50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:19.364015 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f17f7-2aa1-4f26-b19b-afdaae557858 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:19.364642 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f17f7-2aa1-4f26-b19b-afdaae557858 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:19.364922 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f17f7-2aa1-4f26-b19b-afdaae557858 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:19.365193 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f17f7-2aa1-4f26-b19b-afdaae557858 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:19.378734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:19.379647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2432f05-41bd-4d62-a7bd-ba18864dff50 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:19.387048 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed54ace-9bbf-40aa-952e-8ecfe8f24e11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:19.387974 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed54ace-9bbf-40aa-952e-8ecfe8f24e11 service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:19.388875 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed54ace-9bbf-40aa-952e-8ecfe8f24e11 service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:19.388875 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed54ace-9bbf-40aa-952e-8ecfe8f24e11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:19.399625 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed54ace-9bbf-40aa-952e-8ecfe8f24e11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:19.401140 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed54ace-9bbf-40aa-952e-8ecfe8f24e11 service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:19.413402 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 837/1680] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:13:19.549545 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f17f7-2aa1-4f26-b19b-afdaae557858 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:19.550747 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f17f7-2aa1-4f26-b19b-afdaae557858 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:19.569268 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 844/1681] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:19 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:13:19.943144 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed54ace-9bbf-40aa-952e-8ecfe8f24e11 service cinder] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:19.964674 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed54ace-9bbf-40aa-952e-8ecfe8f24e11 service cinder] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:19.986081 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad117f2-cb13-430d-8059-a0cffe187f9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:19.989184 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad117f2-cb13-430d-8059-a0cffe187f9f service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:19.989319 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad117f2-cb13-430d-8059-a0cffe187f9f service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:19.991731 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad117f2-cb13-430d-8059-a0cffe187f9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:19.998277 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-1ad117f2-cb13-430d-8059-a0cffe187f9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:20.170449 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad117f2-cb13-430d-8059-a0cffe187f9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:20.172025 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad117f2-cb13-430d-8059-a0cffe187f9f service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:20.211691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 838/1682] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:13:21.989114 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf90693-057d-4c56-a6d5-26531e36d6a4 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:21.989927 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf90693-057d-4c56-a6d5-26531e36d6a4 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:21.990469 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf90693-057d-4c56-a6d5-26531e36d6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:22.068082 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-ddf90693-057d-4c56-a6d5-26531e36d6a4 None None] MFA Rules not processed for user `e7cb679ed52e42ca83a2084f1ad4bd92`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:22.069668 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ddf90693-057d-4c56-a6d5-26531e36d6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:22.252326 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 845/1683] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:22.270500 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad117f2-cb13-430d-8059-a0cffe187f9f service cinder] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:22.275178 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-1ad117f2-cb13-430d-8059-a0cffe187f9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:22.576387 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad117f2-cb13-430d-8059-a0cffe187f9f service cinder] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:22.602929 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201594e-8ea3-4bc6-a916-a6fde10895d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:22.603634 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201594e-8ea3-4bc6-a916-a6fde10895d3 service nova] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:22.603845 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201594e-8ea3-4bc6-a916-a6fde10895d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:22.604121 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201594e-8ea3-4bc6-a916-a6fde10895d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:22.632239 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201594e-8ea3-4bc6-a916-a6fde10895d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:22.633882 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201594e-8ea3-4bc6-a916-a6fde10895d3 service nova] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:22.657270 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 839/1684] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:13:27.334293 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71cf462-c98f-4cd3-9030-63eb6e0f7274 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:27.335262 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71cf462-c98f-4cd3-9030-63eb6e0f7274 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:27.335549 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71cf462-c98f-4cd3-9030-63eb6e0f7274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:27.381911 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-b71cf462-c98f-4cd3-9030-63eb6e0f7274 None None] MFA Rules not processed for user `e7cb679ed52e42ca83a2084f1ad4bd92`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:27.386993 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b71cf462-c98f-4cd3-9030-63eb6e0f7274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:27.404069 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 846/1685] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:27.788429 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57895950-e0df-4dc8-84ac-1712570d3e51 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:27.789020 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57895950-e0df-4dc8-84ac-1712570d3e51 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:27.789020 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57895950-e0df-4dc8-84ac-1712570d3e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:27.846677 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-57895950-e0df-4dc8-84ac-1712570d3e51 None None] MFA Rules not processed for user `5f2ff5e497b64e49b7ac25a941a06048`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:27.848281 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-57895950-e0df-4dc8-84ac-1712570d3e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:27.911753 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:27.973685 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 840/1686] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:27 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:27.984077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7e454-c2d7-4ac4-882c-b01d42c6d723 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:27.984207 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7e454-c2d7-4ac4-882c-b01d42c6d723 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:27.985125 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7e454-c2d7-4ac4-882c-b01d42c6d723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:28.024502 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-b7f7e454-c2d7-4ac4-882c-b01d42c6d723 None None] MFA Rules not processed for user `5f2ff5e497b64e49b7ac25a941a06048`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:28.026589 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b7f7e454-c2d7-4ac4-882c-b01d42c6d723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:28.039144 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 847/1687] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:27 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:28.050131 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57895950-e0df-4dc8-84ac-1712570d3e51 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:28.073016 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57895950-e0df-4dc8-84ac-1712570d3e51 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:28.083337 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea5898-8f85-4bed-89fc-99ad34cfdcca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:28.091329 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea5898-8f85-4bed-89fc-99ad34cfdcca service nova] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:28.091643 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea5898-8f85-4bed-89fc-99ad34cfdcca service nova] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:28.091928 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea5898-8f85-4bed-89fc-99ad34cfdcca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:28.114261 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea5898-8f85-4bed-89fc-99ad34cfdcca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:28.116124 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea5898-8f85-4bed-89fc-99ad34cfdcca service nova] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:28.135939 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 841/1688] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:28 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:13:28.457024 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee64546-a84d-4bf2-8e04-abc4a1911c59 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:28.458934 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee64546-a84d-4bf2-8e04-abc4a1911c59 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:28.459312 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee64546-a84d-4bf2-8e04-abc4a1911c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:28.599492 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-6ee64546-a84d-4bf2-8e04-abc4a1911c59 None None] MFA Rules not processed for user `0cafcc70e0e74a5fa7c5ce530e0e848f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:28.601768 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-6ee64546-a84d-4bf2-8e04-abc4a1911c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:28.757439 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 848/1689] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:28.805467 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898d63c-bfb9-4ce4-9904-0a6d0eb29f26 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:28.805932 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898d63c-bfb9-4ce4-9904-0a6d0eb29f26 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:28.806433 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898d63c-bfb9-4ce4-9904-0a6d0eb29f26 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:28.808617 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 842/1690] 10.222.0.42 () {54 vars in 912 bytes} [Mon Sep 12 14:13:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:13:28.822739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c003c-cafd-4bd1-8bde-92523b20fae1 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:28.823106 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c003c-cafd-4bd1-8bde-92523b20fae1 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:28.823368 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c003c-cafd-4bd1-8bde-92523b20fae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:28.841287 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e939c-1415-4a30-be99-0c044beba002 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:28.841573 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e939c-1415-4a30-be99-0c044beba002 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:28.842078 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e939c-1415-4a30-be99-0c044beba002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:28.890125 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:28.940615 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-c21e939c-1415-4a30-be99-0c044beba002 None None] MFA Rules not processed for user `fff8e93029f447bb8294d9b516eb407f`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:28.957316 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-c21e939c-1415-4a30-be99-0c044beba002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:28.988840 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-390c003c-cafd-4bd1-8bde-92523b20fae1 None None] MFA Rules not processed for user `865b1647468c4efe8efe4febe2203a58`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:28.990246 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-390c003c-cafd-4bd1-8bde-92523b20fae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:29.135561 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 849/1691] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Sep 12 14:13:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:29.138283 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d37a233-00a4-47d3-a1f5-4aae5abd8632 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:29.138661 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d37a233-00a4-47d3-a1f5-4aae5abd8632 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:29.138846 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d37a233-00a4-47d3-a1f5-4aae5abd8632 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:29.144731 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 850/1692] 10.222.0.42 () {54 vars in 912 bytes} [Mon Sep 12 14:13:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:13:29.155159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 843/1693] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:29.163994 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d37a233-00a4-47d3-a1f5-4aae5abd8632 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:29.165855 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-8d37a233-00a4-47d3-a1f5-4aae5abd8632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:29.171417 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be451633-ac83-4516-a558-798d927adde8 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:29.172483 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be451633-ac83-4516-a558-798d927adde8 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:29.174257 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be451633-ac83-4516-a558-798d927adde8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:29.215601 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d37a233-00a4-47d3-a1f5-4aae5abd8632 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:29.227524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825aa6cb-f3d0-4ace-98f4-0d0b4000c724 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:29.228347 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825aa6cb-f3d0-4ace-98f4-0d0b4000c724 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:29.228927 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825aa6cb-f3d0-4ace-98f4-0d0b4000c724 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:29.229325 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825aa6cb-f3d0-4ace-98f4-0d0b4000c724 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:29.243306 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-be451633-ac83-4516-a558-798d927adde8 None None] MFA Rules not processed for user `865b1647468c4efe8efe4febe2203a58`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:29.266616 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-be451633-ac83-4516-a558-798d927adde8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:29.269228 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825aa6cb-f3d0-4ace-98f4-0d0b4000c724 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:29.284271 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825aa6cb-f3d0-4ace-98f4-0d0b4000c724 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:29.306922 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 844/1694] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Sep 12 14:13:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:29.308353 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915ee01-1bcc-4d31-813a-5386a0878e01 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:29.308635 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915ee01-1bcc-4d31-813a-5386a0878e01 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:29.308902 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915ee01-1bcc-4d31-813a-5386a0878e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:29.318632 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 851/1695] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:13:29.319542 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825aa6cb-f3d0-4ace-98f4-0d0b4000c724 service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:29.364433 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825aa6cb-f3d0-4ace-98f4-0d0b4000c724 service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:29.389289 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085041dc-79ad-461b-b483-4a403da5fbae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:29.389972 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085041dc-79ad-461b-b483-4a403da5fbae service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:29.390251 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085041dc-79ad-461b-b483-4a403da5fbae service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:29.390522 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085041dc-79ad-461b-b483-4a403da5fbae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:29.415766 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085041dc-79ad-461b-b483-4a403da5fbae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:29.422556 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085041dc-79ad-461b-b483-4a403da5fbae service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:29.434895 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-7915ee01-1bcc-4d31-813a-5386a0878e01 None None] MFA Rules not processed for user `1c6cd05ddb3d4d14b2a18bc45401e73d`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:29.436369 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-7915ee01-1bcc-4d31-813a-5386a0878e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:29.454152 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 852/1696] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:13:29.645755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 845/1697] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:29.648757 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9b562-69b4-42f8-9a16-88ee79fbf50c None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:29.649223 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9b562-69b4-42f8-9a16-88ee79fbf50c None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:29.649332 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9b562-69b4-42f8-9a16-88ee79fbf50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:29.704638 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-e8a9b562-69b4-42f8-9a16-88ee79fbf50c None None] MFA Rules not processed for user `1c6cd05ddb3d4d14b2a18bc45401e73d`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:29.711213 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e8a9b562-69b4-42f8-9a16-88ee79fbf50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:29.743535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 853/1698] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:30.345322 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7915ee01-1bcc-4d31-813a-5386a0878e01 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:30.399993 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9b562-69b4-42f8-9a16-88ee79fbf50c None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:30.401689 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e8a9b562-69b4-42f8-9a16-88ee79fbf50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:30.415240 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7915ee01-1bcc-4d31-813a-5386a0878e01 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:30.433955 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd549d7-8293-4e1b-b5d8-008e8dd030ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:30.435218 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd549d7-8293-4e1b-b5d8-008e8dd030ea service nova] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:30.437179 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd549d7-8293-4e1b-b5d8-008e8dd030ea service nova] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:30.441083 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd549d7-8293-4e1b-b5d8-008e8dd030ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:30.482299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9b562-69b4-42f8-9a16-88ee79fbf50c None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:30.498928 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd549d7-8293-4e1b-b5d8-008e8dd030ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:30.502279 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd549d7-8293-4e1b-b5d8-008e8dd030ea service nova] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:30.515348 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09fc14c-af41-44a3-be2c-78313d4a82c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:30.519843 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09fc14c-af41-44a3-be2c-78313d4a82c7 service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:30.520380 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09fc14c-af41-44a3-be2c-78313d4a82c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:30.521131 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09fc14c-af41-44a3-be2c-78313d4a82c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:30.522611 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 846/1699] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:30 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:13:30.523717 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd549d7-8293-4e1b-b5d8-008e8dd030ea service nova] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:30.526339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-bbd549d7-8293-4e1b-b5d8-008e8dd030ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:30.550886 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09fc14c-af41-44a3-be2c-78313d4a82c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:30.552539 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09fc14c-af41-44a3-be2c-78313d4a82c7 service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:30.563045 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 854/1700] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:13:30.576622 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430e166-74c6-43db-979a-9456e51241de None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:30.577214 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430e166-74c6-43db-979a-9456e51241de None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:30.577725 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430e166-74c6-43db-979a-9456e51241de None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:30.579688 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:30.585955 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 855/1701] 10.222.0.42 () {54 vars in 961 bytes} [Mon Sep 12 14:13:30 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:13:30.589232 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eddd00b-e4aa-4657-a5f0-1f05f3c2b98f None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:30.593925 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eddd00b-e4aa-4657-a5f0-1f05f3c2b98f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:30.603216 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eddd00b-e4aa-4657-a5f0-1f05f3c2b98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:30.640495 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-5eddd00b-e4aa-4657-a5f0-1f05f3c2b98f None None] MFA Rules not processed for user `ce508bb9aa164a7fb1831a9b5ff5b932`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:30.643861 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-5eddd00b-e4aa-4657-a5f0-1f05f3c2b98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:30.674155 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd549d7-8293-4e1b-b5d8-008e8dd030ea service nova] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:30.680684 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 856/1702] 10.222.0.42 () {58 vars in 1058 bytes} [Mon Sep 12 14:13:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:30.683031 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eddd00b-e4aa-4657-a5f0-1f05f3c2b98f None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:30.683406 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7216f66-4ec7-4c82-8016-c05f4f8cd0f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce508bb9aa164a7fb1831a9b5ff5b932', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:30.684273 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7216f66-4ec7-4c82-8016-c05f4f8cd0f0 service placement] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:30.684773 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7216f66-4ec7-4c82-8016-c05f4f8cd0f0 service placement] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:30.685215 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7216f66-4ec7-4c82-8016-c05f4f8cd0f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:30.687886 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-c7216f66-4ec7-4c82-8016-c05f4f8cd0f0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:30.708034 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7216f66-4ec7-4c82-8016-c05f4f8cd0f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:30.710690 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7216f66-4ec7-4c82-8016-c05f4f8cd0f0 service placement] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:30.724524 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 847/1703] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:13:30.725466 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7216f66-4ec7-4c82-8016-c05f4f8cd0f0 service placement] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:30.726125 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eddd00b-e4aa-4657-a5f0-1f05f3c2b98f None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:30.738108 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7216f66-4ec7-4c82-8016-c05f4f8cd0f0 service placement] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:30.740869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d6578-0203-449f-9091-02a832e287e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:30.744524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d6578-0203-449f-9091-02a832e287e5 service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:30.744994 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d6578-0203-449f-9091-02a832e287e5 service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:30.745690 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0fc492-2c0c-4b69-aa0d-772848a1f436 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce508bb9aa164a7fb1831a9b5ff5b932', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:30.746282 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d6578-0203-449f-9091-02a832e287e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:30.746552 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fc492-2c0c-4b69-aa0d-772848a1f436 service placement] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:30.747012 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fc492-2c0c-4b69-aa0d-772848a1f436 service placement] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:30.747379 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fc492-2c0c-4b69-aa0d-772848a1f436 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:30.763777 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0fc492-2c0c-4b69-aa0d-772848a1f436 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:30.765833 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0fc492-2c0c-4b69-aa0d-772848a1f436 service placement] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:30.769943 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3d6578-0203-449f-9091-02a832e287e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:30.775741 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3d6578-0203-449f-9091-02a832e287e5 service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:30.780364 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 848/1704] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:13:30.801487 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 857/1705] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:13:31.008332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382c82b-a5f8-4ae1-8cb4-3ce1cf2ad693 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:31.008733 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382c82b-a5f8-4ae1-8cb4-3ce1cf2ad693 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:31.010017 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382c82b-a5f8-4ae1-8cb4-3ce1cf2ad693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:31.084203 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-8382c82b-a5f8-4ae1-8cb4-3ce1cf2ad693 None None] MFA Rules not processed for user `5f2ff5e497b64e49b7ac25a941a06048`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:31.085515 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-8382c82b-a5f8-4ae1-8cb4-3ce1cf2ad693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:31.130778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 849/1706] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:31 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:32.534759 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d6578-0203-449f-9091-02a832e287e5 service nova] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:32.854441 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d6578-0203-449f-9091-02a832e287e5 service nova] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:32.880225 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c51492-790c-4ce8-8292-d43d449ffcc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:32.880892 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c51492-790c-4ce8-8292-d43d449ffcc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:32.886060 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c51492-790c-4ce8-8292-d43d449ffcc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:32.888208 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c51492-790c-4ce8-8292-d43d449ffcc5 admin admin] PATH_INFO: `/v3/users/6c5b60e014114e72afd135da80c158ae` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:32.892680 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c51492-790c-4ce8-8292-d43d449ffcc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c5b60e014114e72afd135da80c158ae)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:32.902942 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c51492-790c-4ce8-8292-d43d449ffcc5 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:33.128572 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:33.167877 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-36c51492-790c-4ce8-8292-d43d449ffcc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c5b60e014114e72afd135da80c158ae'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:33.223455 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-36c51492-790c-4ce8-8292-d43d449ffcc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c5b60e014114e72afd135da80c158ae'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:33.224244 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-36c51492-790c-4ce8-8292-d43d449ffcc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c5b60e014114e72afd135da80c158ae'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:33.229914 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-36c51492-790c-4ce8-8292-d43d449ffcc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c5b60e014114e72afd135da80c158ae'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:33.292416 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-36c51492-790c-4ce8-8292-d43d449ffcc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c5b60e014114e72afd135da80c158ae'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:33.449541 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 858/1707] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:13:32 2022] DELETE /identity/v3/users/6c5b60e014114e72afd135da80c158ae => generated 0 bytes in 917 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:33.454318 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e4336-80ad-4e88-92a9-3166d3305634 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:33.455314 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e4336-80ad-4e88-92a9-3166d3305634 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:33.455423 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e4336-80ad-4e88-92a9-3166d3305634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:33.536101 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-921e4336-80ad-4e88-92a9-3166d3305634 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:33.537345 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-921e4336-80ad-4e88-92a9-3166d3305634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:33.799552 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 850/1708] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:33.873130 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c51492-790c-4ce8-8292-d43d449ffcc5 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:33.875223 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-36c51492-790c-4ce8-8292-d43d449ffcc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:33.936081 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e4336-80ad-4e88-92a9-3166d3305634 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:33.941086 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-921e4336-80ad-4e88-92a9-3166d3305634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:33.966557 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e4336-80ad-4e88-92a9-3166d3305634 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:33.984303 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec23c2e-8a3b-45f8-9bfe-29b129ba2f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:33.986454 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec23c2e-8a3b-45f8-9bfe-29b129ba2f65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:33.986909 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec23c2e-8a3b-45f8-9bfe-29b129ba2f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:33.988173 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec23c2e-8a3b-45f8-9bfe-29b129ba2f65 admin admin] PATH_INFO: `/v3/users/287bcbae81f34d5687dea75ff9569eda` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:33.991311 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec23c2e-8a3b-45f8-9bfe-29b129ba2f65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=287bcbae81f34d5687dea75ff9569eda)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:33.997086 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec23c2e-8a3b-45f8-9bfe-29b129ba2f65 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:34.038547 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c51492-790c-4ce8-8292-d43d449ffcc5 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:34.049439 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4758e64-8c92-4c7f-b7e3-0da1507b2faf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:34.050307 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4758e64-8c92-4c7f-b7e3-0da1507b2faf service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:34.050752 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4758e64-8c92-4c7f-b7e3-0da1507b2faf service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:34.051177 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4758e64-8c92-4c7f-b7e3-0da1507b2faf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:34.091009 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4758e64-8c92-4c7f-b7e3-0da1507b2faf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:34.103227 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4758e64-8c92-4c7f-b7e3-0da1507b2faf service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:34.146574 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 859/1709] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:13:34.149829 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-dec23c2e-8a3b-45f8-9bfe-29b129ba2f65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '287bcbae81f34d5687dea75ff9569eda'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:34.207548 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-dec23c2e-8a3b-45f8-9bfe-29b129ba2f65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '287bcbae81f34d5687dea75ff9569eda'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:34.208459 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-dec23c2e-8a3b-45f8-9bfe-29b129ba2f65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '287bcbae81f34d5687dea75ff9569eda'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:34.209911 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-dec23c2e-8a3b-45f8-9bfe-29b129ba2f65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '287bcbae81f34d5687dea75ff9569eda'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:34.231112 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-dec23c2e-8a3b-45f8-9bfe-29b129ba2f65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '287bcbae81f34d5687dea75ff9569eda'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:34.277962 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 851/1710] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:13:33 2022] DELETE /identity/v3/users/287bcbae81f34d5687dea75ff9569eda => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:34.282278 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee7bbf-0192-4693-a768-b61456cfc450 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:34.283493 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee7bbf-0192-4693-a768-b61456cfc450 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:34.283764 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee7bbf-0192-4693-a768-b61456cfc450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:34.337306 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-94ee7bbf-0192-4693-a768-b61456cfc450 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:34.339394 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-94ee7bbf-0192-4693-a768-b61456cfc450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:34.485263 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 860/1711] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:34.540512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec23c2e-8a3b-45f8-9bfe-29b129ba2f65 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:34.542101 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-dec23c2e-8a3b-45f8-9bfe-29b129ba2f65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:34.577000 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:34.761854 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec23c2e-8a3b-45f8-9bfe-29b129ba2f65 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:34.774232 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9947a1eb-3278-49ff-b4be-f49f2505c307 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:34.774897 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947a1eb-3278-49ff-b4be-f49f2505c307 service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:34.775326 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947a1eb-3278-49ff-b4be-f49f2505c307 service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:34.775894 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947a1eb-3278-49ff-b4be-f49f2505c307 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:34.815817 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9947a1eb-3278-49ff-b4be-f49f2505c307 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:34.821582 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9947a1eb-3278-49ff-b4be-f49f2505c307 service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:34.846071 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 852/1712] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:13:35.159958 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee7bbf-0192-4693-a768-b61456cfc450 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:35.162411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-94ee7bbf-0192-4693-a768-b61456cfc450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:35.196427 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee7bbf-0192-4693-a768-b61456cfc450 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:35.209680 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ff9c72-14c9-4913-8f3e-80a17b663e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:35.211803 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff9c72-14c9-4913-8f3e-80a17b663e98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:35.213993 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff9c72-14c9-4913-8f3e-80a17b663e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:35.214589 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff9c72-14c9-4913-8f3e-80a17b663e98 admin admin] PATH_INFO: `/v3/projects/f7958feaecbe4664826f14b8717e0525` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:35.221382 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ff9c72-14c9-4913-8f3e-80a17b663e98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7958feaecbe4664826f14b8717e0525)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:35.228940 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ff9c72-14c9-4913-8f3e-80a17b663e98 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:35.365702 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-49ff9c72-14c9-4913-8f3e-80a17b663e98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7958feaecbe4664826f14b8717e0525', 'request_id': 'req-49ff9c72-14c9-4913-8f3e-80a17b663e98'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:35.388945 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-49ff9c72-14c9-4913-8f3e-80a17b663e98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7958feaecbe4664826f14b8717e0525', 'request_id': 'req-49ff9c72-14c9-4913-8f3e-80a17b663e98'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:35.391021 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:35.401200 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-49ff9c72-14c9-4913-8f3e-80a17b663e98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7958feaecbe4664826f14b8717e0525', 'request_id': 'req-49ff9c72-14c9-4913-8f3e-80a17b663e98'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:35.444034 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-49ff9c72-14c9-4913-8f3e-80a17b663e98 admin admin] The token cache is being invalidate because project f7958feaecbe4664826f14b8717e0525 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:13:35.444775 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-49ff9c72-14c9-4913-8f3e-80a17b663e98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:35.447306 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-49ff9c72-14c9-4913-8f3e-80a17b663e98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:35.452603 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 861/1713] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:13:35 2022] DELETE /identity/v3/projects/f7958feaecbe4664826f14b8717e0525 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:35.458524 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ef581-fd26-4f27-b2b4-5b29cd92c455 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:35.458784 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ef581-fd26-4f27-b2b4-5b29cd92c455 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:35.459609 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ef581-fd26-4f27-b2b4-5b29cd92c455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:35.514198 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-0a1ef581-fd26-4f27-b2b4-5b29cd92c455 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:35.515815 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-0a1ef581-fd26-4f27-b2b4-5b29cd92c455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:35.671022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 853/1714] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:35.678695 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955a639-025e-46d7-b786-4ca9018747a7 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:35.679018 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955a639-025e-46d7-b786-4ca9018747a7 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:35.679494 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955a639-025e-46d7-b786-4ca9018747a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:35.729012 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b523cf9-cf52-45b2-a404-6db657dd4210 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:35.729299 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b523cf9-cf52-45b2-a404-6db657dd4210 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:35.729624 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b523cf9-cf52-45b2-a404-6db657dd4210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:35.751765 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-e955a639-025e-46d7-b786-4ca9018747a7 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:35.756905 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e955a639-025e-46d7-b786-4ca9018747a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:35.788984 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 862/1715] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:35.790363 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12497e5e-9b7a-4aea-b7ef-8b1a95203668 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:35.793132 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12497e5e-9b7a-4aea-b7ef-8b1a95203668 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:35.793281 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12497e5e-9b7a-4aea-b7ef-8b1a95203668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:35.856296 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-12497e5e-9b7a-4aea-b7ef-8b1a95203668 None None] MFA Rules not processed for user `fff8e93029f447bb8294d9b516eb407f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:35.856296 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-12497e5e-9b7a-4aea-b7ef-8b1a95203668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:35.864296 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-3b523cf9-cf52-45b2-a404-6db657dd4210 None None] MFA Rules not processed for user `1405e758a3af4e8ba659f498cd28ace1`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:35.869405 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-3b523cf9-cf52-45b2-a404-6db657dd4210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:36.089972 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 863/1716] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:36.098575 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 854/1717] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Sep 12 14:13:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:13:36.119633 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12497e5e-9b7a-4aea-b7ef-8b1a95203668 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:36.168649 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12497e5e-9b7a-4aea-b7ef-8b1a95203668 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:36.187154 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4026329-2cae-4b60-b789-257463d0a454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:36.187816 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4026329-2cae-4b60-b789-257463d0a454 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:36.190403 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4026329-2cae-4b60-b789-257463d0a454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:36.190611 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4026329-2cae-4b60-b789-257463d0a454 admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:36.192679 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4026329-2cae-4b60-b789-257463d0a454 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:13:36.207462 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4026329-2cae-4b60-b789-257463d0a454 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:36.229661 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4026329-2cae-4b60-b789-257463d0a454 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:36.283595 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 864/1718] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:13:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:13:36.284977 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b523cf9-cf52-45b2-a404-6db657dd4210 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:36.299885 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b523cf9-cf52-45b2-a404-6db657dd4210 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:36.317052 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e35837-6721-42ae-92f8-1b0678907c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:36.319586 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e35837-6721-42ae-92f8-1b0678907c53 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:36.320062 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e35837-6721-42ae-92f8-1b0678907c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:36.320541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e35837-6721-42ae-92f8-1b0678907c53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:36.321484 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e35837-6721-42ae-92f8-1b0678907c53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:36.328383 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e35837-6721-42ae-92f8-1b0678907c53 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:36.384031 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 855/1719] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:13:36 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:36.388189 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4026329-2cae-4b60-b789-257463d0a454 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:36.389994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e35837-6721-42ae-92f8-1b0678907c53 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:36.390605 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d4026329-2cae-4b60-b789-257463d0a454 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:36.396947 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:36.525873 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e35837-6721-42ae-92f8-1b0678907c53 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:36.537914 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9c82b-38ce-4340-bb88-f29462d19198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:36.538611 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9c82b-38ce-4340-bb88-f29462d19198 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:36.538925 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9c82b-38ce-4340-bb88-f29462d19198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:36.539444 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9c82b-38ce-4340-bb88-f29462d19198 admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:36.540444 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f9c82b-38ce-4340-bb88-f29462d19198 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:36.556332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f9c82b-38ce-4340-bb88-f29462d19198 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:36.606751 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4026329-2cae-4b60-b789-257463d0a454 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:36.650857 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 856/1720] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:13:36 2022] POST /identity/v3/users => generated 287 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:36.656967 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9c82b-38ce-4340-bb88-f29462d19198 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:36.698276 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9c82b-38ce-4340-bb88-f29462d19198 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:36.718119 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce0f49-c84a-49ab-82ed-c376c48c22a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:36.719069 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce0f49-c84a-49ab-82ed-c376c48c22a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:36.719363 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce0f49-c84a-49ab-82ed-c376c48c22a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:36.719639 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce0f49-c84a-49ab-82ed-c376c48c22a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:36.720850 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ce0f49-c84a-49ab-82ed-c376c48c22a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:36.724847 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ce0f49-c84a-49ab-82ed-c376c48c22a4 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:36.757634 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 857/1721] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:36 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:36.761919 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce0f49-c84a-49ab-82ed-c376c48c22a4 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:36.792544 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce0f49-c84a-49ab-82ed-c376c48c22a4 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:36.812392 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054957f-e622-4ae3-94bd-211cb35f1e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:36.813243 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054957f-e622-4ae3-94bd-211cb35f1e76 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:36.813581 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054957f-e622-4ae3-94bd-211cb35f1e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:36.814935 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054957f-e622-4ae3-94bd-211cb35f1e76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:36.826501 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054957f-e622-4ae3-94bd-211cb35f1e76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:36.828570 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054957f-e622-4ae3-94bd-211cb35f1e76 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:36.853705 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 858/1722] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:36 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:36.857832 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054957f-e622-4ae3-94bd-211cb35f1e76 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:36.868858 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:36.884398 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054957f-e622-4ae3-94bd-211cb35f1e76 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:36.899122 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f94c2-37fa-4db3-98ef-e9b5d7bd0d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:36.900017 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f94c2-37fa-4db3-98ef-e9b5d7bd0d09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:36.902564 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f94c2-37fa-4db3-98ef-e9b5d7bd0d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:36.903042 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f94c2-37fa-4db3-98ef-e9b5d7bd0d09 admin admin] PATH_INFO: `/v3/projects/7e3af2264bad4107928e28327806d8a8/users/a4d1432f4d5349048189f1017199691e/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:36.911378 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a970a85a-389e-43c6-a879-901cd86b5de2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a87ea9c3c5409b97a35c4857ae3ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:36.921708 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970a85a-389e-43c6-a879-901cd86b5de2 service swift] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:36.923716 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970a85a-389e-43c6-a879-901cd86b5de2 service swift] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:36.924468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970a85a-389e-43c6-a879-901cd86b5de2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:36.951765 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045f94c2-37fa-4db3-98ef-e9b5d7bd0d09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e3af2264bad4107928e28327806d8a8, user_id=a4d1432f4d5349048189f1017199691e, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:36.957857 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045f94c2-37fa-4db3-98ef-e9b5d7bd0d09 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:36.997341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 859/1723] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:13:36 2022] PUT /identity/v3/projects/7e3af2264bad4107928e28327806d8a8/users/a4d1432f4d5349048189f1017199691e/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:37.008944 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2ceab-a304-4c3c-ab90-4814ee9b6058 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:37.009492 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2ceab-a304-4c3c-ab90-4814ee9b6058 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:37.009994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2ceab-a304-4c3c-ab90-4814ee9b6058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:37.071069 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-82e2ceab-a304-4c3c-ab90-4814ee9b6058 None None] MFA Rules not processed for user `a4d1432f4d5349048189f1017199691e`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:37.074517 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-82e2ceab-a304-4c3c-ab90-4814ee9b6058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:37.163236 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a970a85a-389e-43c6-a879-901cd86b5de2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:37.171034 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a970a85a-389e-43c6-a879-901cd86b5de2 service swift] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:37.195406 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 865/1724] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Sep 12 14:13:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 808 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 14:13:37.197378 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eda47f-7f9b-4eba-be7f-52eca640efe9 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:37.197582 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eda47f-7f9b-4eba-be7f-52eca640efe9 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:37.197898 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eda47f-7f9b-4eba-be7f-52eca640efe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:37.260671 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-29eda47f-7f9b-4eba-be7f-52eca640efe9 None None] MFA Rules not processed for user `0cafcc70e0e74a5fa7c5ce530e0e848f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:37.292330 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-29eda47f-7f9b-4eba-be7f-52eca640efe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:37.431175 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 860/1725] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 425 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:37.436114 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2ceab-a304-4c3c-ab90-4814ee9b6058 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:37.457782 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-82e2ceab-a304-4c3c-ab90-4814ee9b6058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:37.670398 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 866/1726] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 474 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:37.867512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:38.055706 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2ceab-a304-4c3c-ab90-4814ee9b6058 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:38.083894 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6e4dd-5e23-41e7-8a58-31e8fe0b5cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:38.088455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6e4dd-5e23-41e7-8a58-31e8fe0b5cf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:38.088784 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6e4dd-5e23-41e7-8a58-31e8fe0b5cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:38.089062 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6e4dd-5e23-41e7-8a58-31e8fe0b5cf9 admin admin] PATH_INFO: `/v3/projects/c20b4e9d9c6e46bbbbeb56e7ac472859` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:38.094826 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d6e4dd-5e23-41e7-8a58-31e8fe0b5cf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c20b4e9d9c6e46bbbbeb56e7ac472859)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:38.102029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d6e4dd-5e23-41e7-8a58-31e8fe0b5cf9 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:38.224869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eda47f-7f9b-4eba-be7f-52eca640efe9 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:38.226430 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-29eda47f-7f9b-4eba-be7f-52eca640efe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:38.280903 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eda47f-7f9b-4eba-be7f-52eca640efe9 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:38.305653 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c1b3ec-7d89-4e85-9214-b5e379efed10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:38.309096 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1b3ec-7d89-4e85-9214-b5e379efed10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:38.312727 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1b3ec-7d89-4e85-9214-b5e379efed10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:38.313007 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1b3ec-7d89-4e85-9214-b5e379efed10 admin admin] PATH_INFO: `/v3/users/3dd645b707b34755b0fd9b88ea17540b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:38.317287 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c1b3ec-7d89-4e85-9214-b5e379efed10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dd645b707b34755b0fd9b88ea17540b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:38.328191 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c1b3ec-7d89-4e85-9214-b5e379efed10 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:38.373707 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-48d6e4dd-5e23-41e7-8a58-31e8fe0b5cf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c20b4e9d9c6e46bbbbeb56e7ac472859', 'request_id': 'req-48d6e4dd-5e23-41e7-8a58-31e8fe0b5cf9'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:38.393806 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-48d6e4dd-5e23-41e7-8a58-31e8fe0b5cf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c20b4e9d9c6e46bbbbeb56e7ac472859', 'request_id': 'req-48d6e4dd-5e23-41e7-8a58-31e8fe0b5cf9'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:38.408458 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-48d6e4dd-5e23-41e7-8a58-31e8fe0b5cf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c20b4e9d9c6e46bbbbeb56e7ac472859', 'request_id': 'req-48d6e4dd-5e23-41e7-8a58-31e8fe0b5cf9'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:38.431641 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-48d6e4dd-5e23-41e7-8a58-31e8fe0b5cf9 admin admin] The token cache is being invalidate because project c20b4e9d9c6e46bbbbeb56e7ac472859 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:13:38.431832 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-48d6e4dd-5e23-41e7-8a58-31e8fe0b5cf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:38.433529 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-48d6e4dd-5e23-41e7-8a58-31e8fe0b5cf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:38.436913 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 861/1727] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:13:37 2022] DELETE /identity/v3/projects/c20b4e9d9c6e46bbbbeb56e7ac472859 => generated 0 bytes in 1001 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:38.442809 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6e4dd-5e23-41e7-8a58-31e8fe0b5cf9 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:38.446397 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-48d6e4dd-5e23-41e7-8a58-31e8fe0b5cf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:38.507576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a8c1b3ec-7d89-4e85-9214-b5e379efed10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dd645b707b34755b0fd9b88ea17540b'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:38.537161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a8c1b3ec-7d89-4e85-9214-b5e379efed10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dd645b707b34755b0fd9b88ea17540b'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:38.538898 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a8c1b3ec-7d89-4e85-9214-b5e379efed10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dd645b707b34755b0fd9b88ea17540b'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:38.540724 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a8c1b3ec-7d89-4e85-9214-b5e379efed10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dd645b707b34755b0fd9b88ea17540b'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:38.562323 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a8c1b3ec-7d89-4e85-9214-b5e379efed10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dd645b707b34755b0fd9b88ea17540b'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:38.651392 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 867/1728] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:13:38 2022] DELETE /identity/v3/users/3dd645b707b34755b0fd9b88ea17540b => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:38.652773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf8397-140b-4e52-87a6-2e0ed4c9799a None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:38.653060 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf8397-140b-4e52-87a6-2e0ed4c9799a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:38.653334 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf8397-140b-4e52-87a6-2e0ed4c9799a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:38.713315 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-49cf8397-140b-4e52-87a6-2e0ed4c9799a None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:38.721482 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6e4dd-5e23-41e7-8a58-31e8fe0b5cf9 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:38.722459 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-49cf8397-140b-4e52-87a6-2e0ed4c9799a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:38.738772 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:38.887654 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 868/1729] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:38.912677 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf8397-140b-4e52-87a6-2e0ed4c9799a None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:38.914209 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-49cf8397-140b-4e52-87a6-2e0ed4c9799a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:38.924032 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b0249-2490-43b5-bbb2-7222c1bc4348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:38.924563 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b0249-2490-43b5-bbb2-7222c1bc4348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:38.925097 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b0249-2490-43b5-bbb2-7222c1bc4348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:38.925485 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b0249-2490-43b5-bbb2-7222c1bc4348 admin admin] PATH_INFO: `/v3/users/ada88975e82445c596ea4eca15b7794d` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:38.926976 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913b0249-2490-43b5-bbb2-7222c1bc4348 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ada88975e82445c596ea4eca15b7794d)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:38.929051 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913b0249-2490-43b5-bbb2-7222c1bc4348 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:38.983409 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:39.068270 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-913b0249-2490-43b5-bbb2-7222c1bc4348 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ada88975e82445c596ea4eca15b7794d'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:39.079890 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-913b0249-2490-43b5-bbb2-7222c1bc4348 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ada88975e82445c596ea4eca15b7794d'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:39.081065 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-913b0249-2490-43b5-bbb2-7222c1bc4348 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ada88975e82445c596ea4eca15b7794d'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:39.087409 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-913b0249-2490-43b5-bbb2-7222c1bc4348 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ada88975e82445c596ea4eca15b7794d'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:39.128428 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-913b0249-2490-43b5-bbb2-7222c1bc4348 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ada88975e82445c596ea4eca15b7794d'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:39.149171 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf8397-140b-4e52-87a6-2e0ed4c9799a None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:39.163630 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c5a20-e0bd-42c5-8afe-74b490d36491 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:39.164273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c5a20-e0bd-42c5-8afe-74b490d36491 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:39.164546 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c5a20-e0bd-42c5-8afe-74b490d36491 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:39.164818 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c5a20-e0bd-42c5-8afe-74b490d36491 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:39.173765 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-048c5a20-e0bd-42c5-8afe-74b490d36491 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:39.197103 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 862/1730] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:13:38 2022] DELETE /identity/v3/users/ada88975e82445c596ea4eca15b7794d => generated 0 bytes in 752 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:39.198876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64503ced-1c05-4f29-bf01-52bdcdc7e5cf None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:39.198876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64503ced-1c05-4f29-bf01-52bdcdc7e5cf None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:39.198876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64503ced-1c05-4f29-bf01-52bdcdc7e5cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:39.203492 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048c5a20-e0bd-42c5-8afe-74b490d36491 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:39.223415 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048c5a20-e0bd-42c5-8afe-74b490d36491 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:39.254130 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 869/1731] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:13:39.256848 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c5a20-e0bd-42c5-8afe-74b490d36491 service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:39.259962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-048c5a20-e0bd-42c5-8afe-74b490d36491 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:39.272077 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-64503ced-1c05-4f29-bf01-52bdcdc7e5cf None None] MFA Rules not processed for user `865b1647468c4efe8efe4febe2203a58`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:39.273325 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-64503ced-1c05-4f29-bf01-52bdcdc7e5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:39.458176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 863/1732] 10.222.0.23 () {58 vars in 1015 bytes} [Mon Sep 12 14:13:39 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:39.492127 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c5a20-e0bd-42c5-8afe-74b490d36491 service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:39.502520 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64503ced-1c05-4f29-bf01-52bdcdc7e5cf None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:39.506250 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-64503ced-1c05-4f29-bf01-52bdcdc7e5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:39.531966 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64503ced-1c05-4f29-bf01-52bdcdc7e5cf None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:39.542170 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48152e6d-4121-4985-a44d-cd7ffa112e05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:39.543036 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48152e6d-4121-4985-a44d-cd7ffa112e05 service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:39.543359 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48152e6d-4121-4985-a44d-cd7ffa112e05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:39.544958 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48152e6d-4121-4985-a44d-cd7ffa112e05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:39.571927 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48152e6d-4121-4985-a44d-cd7ffa112e05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:39.573771 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48152e6d-4121-4985-a44d-cd7ffa112e05 service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:39.592097 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 864/1733] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:13:39.704616 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:39.776021 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb91ea-c3c6-466d-bc51-85bedde3466d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:39.776668 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb91ea-c3c6-466d-bc51-85bedde3466d service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:39.776954 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb91ea-c3c6-466d-bc51-85bedde3466d service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:39.777228 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb91ea-c3c6-466d-bc51-85bedde3466d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:39.782886 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d9eb91ea-c3c6-466d-bc51-85bedde3466d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:40.021871 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb91ea-c3c6-466d-bc51-85bedde3466d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:40.029016 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb91ea-c3c6-466d-bc51-85bedde3466d service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:40.074388 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 870/1734] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 819 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:13:41.216040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48152e6d-4121-4985-a44d-cd7ffa112e05 service neutron] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:41.218716 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-48152e6d-4121-4985-a44d-cd7ffa112e05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:41.414527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb91ea-c3c6-466d-bc51-85bedde3466d service nova] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:41.420527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d9eb91ea-c3c6-466d-bc51-85bedde3466d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:41.523439 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48152e6d-4121-4985-a44d-cd7ffa112e05 service neutron] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:41.543170 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621fe7f-d81d-42ac-97d2-a1899ea9ae1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:41.545086 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621fe7f-d81d-42ac-97d2-a1899ea9ae1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:41.548474 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621fe7f-d81d-42ac-97d2-a1899ea9ae1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:41.548879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621fe7f-d81d-42ac-97d2-a1899ea9ae1d admin admin] PATH_INFO: `/v3/projects/1aa08aeb75f94056b5530257d1396e91` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:41.554509 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6621fe7f-d81d-42ac-97d2-a1899ea9ae1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1aa08aeb75f94056b5530257d1396e91)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:41.564867 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6621fe7f-d81d-42ac-97d2-a1899ea9ae1d admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:41.656823 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:41.671938 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb91ea-c3c6-466d-bc51-85bedde3466d service nova] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:41.681695 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-6621fe7f-d81d-42ac-97d2-a1899ea9ae1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1aa08aeb75f94056b5530257d1396e91', 'request_id': 'req-6621fe7f-d81d-42ac-97d2-a1899ea9ae1d'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:41.699220 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-6621fe7f-d81d-42ac-97d2-a1899ea9ae1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1aa08aeb75f94056b5530257d1396e91', 'request_id': 'req-6621fe7f-d81d-42ac-97d2-a1899ea9ae1d'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:41.710915 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-6621fe7f-d81d-42ac-97d2-a1899ea9ae1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1aa08aeb75f94056b5530257d1396e91', 'request_id': 'req-6621fe7f-d81d-42ac-97d2-a1899ea9ae1d'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:41.729844 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-6621fe7f-d81d-42ac-97d2-a1899ea9ae1d admin admin] The token cache is being invalidate because project 1aa08aeb75f94056b5530257d1396e91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:13:41.730173 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-6621fe7f-d81d-42ac-97d2-a1899ea9ae1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:41.730934 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-6621fe7f-d81d-42ac-97d2-a1899ea9ae1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:41.735017 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 865/1735] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:13:41 2022] DELETE /identity/v3/projects/1aa08aeb75f94056b5530257d1396e91 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:41.740546 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6f2f6-e1a4-432f-8ab7-7739a1697217 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:41.740952 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6f2f6-e1a4-432f-8ab7-7739a1697217 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:41.741343 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6f2f6-e1a4-432f-8ab7-7739a1697217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:41.766772 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-42f6f2f6-e1a4-432f-8ab7-7739a1697217 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:41.768438 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-42f6f2f6-e1a4-432f-8ab7-7739a1697217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:41.877504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e43e6c-5d7a-4165-9c5a-72d583c8b438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:41.879616 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e43e6c-5d7a-4165-9c5a-72d583c8b438 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:41.880770 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e43e6c-5d7a-4165-9c5a-72d583c8b438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:41.881188 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e43e6c-5d7a-4165-9c5a-72d583c8b438 admin admin] PATH_INFO: `/v3/projects/fed4a4ba0eda499b90fd268cbb2dfa92` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:41.883634 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e43e6c-5d7a-4165-9c5a-72d583c8b438 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fed4a4ba0eda499b90fd268cbb2dfa92)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:41.888232 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e43e6c-5d7a-4165-9c5a-72d583c8b438 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:41.902951 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 866/1736] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:41.910441 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143944f1-d9c2-45f6-8404-48e639626933 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:41.910944 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143944f1-d9c2-45f6-8404-48e639626933 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:41.911322 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143944f1-d9c2-45f6-8404-48e639626933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:41.989920 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-143944f1-d9c2-45f6-8404-48e639626933 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:41.991522 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-143944f1-d9c2-45f6-8404-48e639626933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:42.038127 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 867/1737] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:13:42.043145 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143944f1-d9c2-45f6-8404-48e639626933 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:42.045256 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:42.076805 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143944f1-d9c2-45f6-8404-48e639626933 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:42.077434 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-99e43e6c-5d7a-4165-9c5a-72d583c8b438 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fed4a4ba0eda499b90fd268cbb2dfa92', 'request_id': 'req-99e43e6c-5d7a-4165-9c5a-72d583c8b438'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:42.090821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-99e43e6c-5d7a-4165-9c5a-72d583c8b438 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fed4a4ba0eda499b90fd268cbb2dfa92', 'request_id': 'req-99e43e6c-5d7a-4165-9c5a-72d583c8b438'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:42.102002 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-99e43e6c-5d7a-4165-9c5a-72d583c8b438 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fed4a4ba0eda499b90fd268cbb2dfa92', 'request_id': 'req-99e43e6c-5d7a-4165-9c5a-72d583c8b438'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:42.146380 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-99e43e6c-5d7a-4165-9c5a-72d583c8b438 admin admin] The token cache is being invalidate because project fed4a4ba0eda499b90fd268cbb2dfa92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:13:42.146916 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-99e43e6c-5d7a-4165-9c5a-72d583c8b438 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:42.150172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-99e43e6c-5d7a-4165-9c5a-72d583c8b438 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:42.162565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 871/1738] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:13:41 2022] DELETE /identity/v3/projects/fed4a4ba0eda499b90fd268cbb2dfa92 => generated 0 bytes in 749 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:42.163759 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e43e6c-5d7a-4165-9c5a-72d583c8b438 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:42.167484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-99e43e6c-5d7a-4165-9c5a-72d583c8b438 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:42.324966 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741adb7-ccdb-4752-af03-dfbcb4db58a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:42.328476 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741adb7-ccdb-4752-af03-dfbcb4db58a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:42.330409 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741adb7-ccdb-4752-af03-dfbcb4db58a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:42.330883 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741adb7-ccdb-4752-af03-dfbcb4db58a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:42.331750 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741adb7-ccdb-4752-af03-dfbcb4db58a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:13:42.339117 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741adb7-ccdb-4752-af03-dfbcb4db58a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:42.352007 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741adb7-ccdb-4752-af03-dfbcb4db58a1 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:42.404720 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e43e6c-5d7a-4165-9c5a-72d583c8b438 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:42.412297 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 868/1739] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:13:42.414081 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741adb7-ccdb-4752-af03-dfbcb4db58a1 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:42.422680 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-c741adb7-ccdb-4752-af03-dfbcb4db58a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:42.428105 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832d308-db0a-4d21-a119-137b58945da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:42.428771 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832d308-db0a-4d21-a119-137b58945da6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:42.432882 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832d308-db0a-4d21-a119-137b58945da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:42.433193 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832d308-db0a-4d21-a119-137b58945da6 admin admin] PATH_INFO: `/v3/users/388b52f56dee4ac1ad4007fed6841d0e` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:42.440106 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2832d308-db0a-4d21-a119-137b58945da6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=388b52f56dee4ac1ad4007fed6841d0e)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:42.447282 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2832d308-db0a-4d21-a119-137b58945da6 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:42.485679 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741adb7-ccdb-4752-af03-dfbcb4db58a1 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:42.518887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e5ebc9-0ad9-47e7-ac53-43f557a6e8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:42.519861 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e5ebc9-0ad9-47e7-ac53-43f557a6e8ac admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:42.521415 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e5ebc9-0ad9-47e7-ac53-43f557a6e8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:42.521996 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e5ebc9-0ad9-47e7-ac53-43f557a6e8ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:42.524942 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e5ebc9-0ad9-47e7-ac53-43f557a6e8ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:42.530436 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e5ebc9-0ad9-47e7-ac53-43f557a6e8ac admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:42.634648 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2832d308-db0a-4d21-a119-137b58945da6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '388b52f56dee4ac1ad4007fed6841d0e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:42.663943 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2832d308-db0a-4d21-a119-137b58945da6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '388b52f56dee4ac1ad4007fed6841d0e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:42.664572 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2832d308-db0a-4d21-a119-137b58945da6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '388b52f56dee4ac1ad4007fed6841d0e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:42.666619 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2832d308-db0a-4d21-a119-137b58945da6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '388b52f56dee4ac1ad4007fed6841d0e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:42.671493 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 869/1740] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:13:42 2022] POST /identity/v3/projects => generated 387 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:42.673781 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e5ebc9-0ad9-47e7-ac53-43f557a6e8ac admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:42.691167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-23e5ebc9-0ad9-47e7-ac53-43f557a6e8ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:42.712631 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2832d308-db0a-4d21-a119-137b58945da6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '388b52f56dee4ac1ad4007fed6841d0e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:42.833486 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 872/1741] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:13:42 2022] DELETE /identity/v3/users/388b52f56dee4ac1ad4007fed6841d0e => generated 0 bytes in 667 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:42.850494 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:43.000369 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e5ebc9-0ad9-47e7-ac53-43f557a6e8ac admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:43.164933 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd782ea5-0ec7-4bbf-b032-24f41e440c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:43.165811 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd782ea5-0ec7-4bbf-b032-24f41e440c3f admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:43.166100 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd782ea5-0ec7-4bbf-b032-24f41e440c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:43.166428 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd782ea5-0ec7-4bbf-b032-24f41e440c3f admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:43.172735 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd782ea5-0ec7-4bbf-b032-24f41e440c3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:43.191407 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd782ea5-0ec7-4bbf-b032-24f41e440c3f admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:43.271586 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 870/1742] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:13:42 2022] POST /identity/v3/users => generated 294 bytes in 599 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:43.272137 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832d308-db0a-4d21-a119-137b58945da6 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:43.286581 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832d308-db0a-4d21-a119-137b58945da6 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:43.300372 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b4ee53-9a37-4b23-a370-0a89a8548d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:43.301005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4ee53-9a37-4b23-a370-0a89a8548d17 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:43.301291 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4ee53-9a37-4b23-a370-0a89a8548d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:43.301710 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4ee53-9a37-4b23-a370-0a89a8548d17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:43.302221 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b4ee53-9a37-4b23-a370-0a89a8548d17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:43.303874 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b4ee53-9a37-4b23-a370-0a89a8548d17 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:43.331783 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 873/1743] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:43.333655 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd782ea5-0ec7-4bbf-b032-24f41e440c3f admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:43.355076 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd782ea5-0ec7-4bbf-b032-24f41e440c3f admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:43.372746 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd714e2-50ab-4480-838b-938029210000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:43.378163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd714e2-50ab-4480-838b-938029210000 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:43.378546 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd714e2-50ab-4480-838b-938029210000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:43.381071 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd714e2-50ab-4480-838b-938029210000 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:43.384840 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd714e2-50ab-4480-838b-938029210000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:43.404201 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd714e2-50ab-4480-838b-938029210000 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:43.448623 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 871/1744] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:43.450154 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b4ee53-9a37-4b23-a370-0a89a8548d17 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:43.467624 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b4ee53-9a37-4b23-a370-0a89a8548d17 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:43.486026 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a85a2-ce3d-4ffa-bd57-c3f0b1e50f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:43.488937 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a85a2-ce3d-4ffa-bd57-c3f0b1e50f40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:43.492466 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a85a2-ce3d-4ffa-bd57-c3f0b1e50f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:43.492816 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a85a2-ce3d-4ffa-bd57-c3f0b1e50f40 admin admin] PATH_INFO: `/v3/projects/767d656ee1ef48ed83675e3f0e1769f1/users/cdab6cc4ba5d4a3b999d28252411c6fd/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:43.547206 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814a85a2-ce3d-4ffa-bd57-c3f0b1e50f40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=767d656ee1ef48ed83675e3f0e1769f1, user_id=cdab6cc4ba5d4a3b999d28252411c6fd, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:43.555414 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814a85a2-ce3d-4ffa-bd57-c3f0b1e50f40 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:43.604624 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 874/1745] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:13:43 2022] PUT /identity/v3/projects/767d656ee1ef48ed83675e3f0e1769f1/users/cdab6cc4ba5d4a3b999d28252411c6fd/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:43.615411 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90debda8-118a-4909-b68c-d6d26c645432 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:43.615704 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90debda8-118a-4909-b68c-d6d26c645432 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:43.615986 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90debda8-118a-4909-b68c-d6d26c645432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:43.707574 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-90debda8-118a-4909-b68c-d6d26c645432 None None] MFA Rules not processed for user `cdab6cc4ba5d4a3b999d28252411c6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:43.709493 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-90debda8-118a-4909-b68c-d6d26c645432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:43.938739 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 872/1746] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:43.940342 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a85a2-ce3d-4ffa-bd57-c3f0b1e50f40 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:43.980190 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:44.092359 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a85a2-ce3d-4ffa-bd57-c3f0b1e50f40 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:44.101005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d98b34d-f44f-441f-a8da-e8aecfdfd6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:44.101853 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98b34d-f44f-441f-a8da-e8aecfdfd6bf admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:44.102632 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98b34d-f44f-441f-a8da-e8aecfdfd6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:44.103371 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98b34d-f44f-441f-a8da-e8aecfdfd6bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:44.104185 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d98b34d-f44f-441f-a8da-e8aecfdfd6bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:44.107556 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d98b34d-f44f-441f-a8da-e8aecfdfd6bf admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:44.168678 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 875/1747] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:13:43 2022] POST /identity/v3/projects => generated 387 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:44.172176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90debda8-118a-4909-b68c-d6d26c645432 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:44.187317 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90debda8-118a-4909-b68c-d6d26c645432 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:44.350382 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:44.441595 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5db23e-af67-4000-a172-ff2a15435c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:44.442457 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5db23e-af67-4000-a172-ff2a15435c1a admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:44.442921 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5db23e-af67-4000-a172-ff2a15435c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:44.443362 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5db23e-af67-4000-a172-ff2a15435c1a admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:44.447116 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5db23e-af67-4000-a172-ff2a15435c1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:44.449367 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5db23e-af67-4000-a172-ff2a15435c1a admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:44.522182 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 873/1748] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:13:44 2022] POST /identity/v3/users => generated 300 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:44.526683 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d98b34d-f44f-441f-a8da-e8aecfdfd6bf admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:44.543223 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d98b34d-f44f-441f-a8da-e8aecfdfd6bf admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:44.558751 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42623cc8-0a2b-4b90-a52d-8886dd6c9d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:44.559441 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42623cc8-0a2b-4b90-a52d-8886dd6c9d6a admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:44.559732 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42623cc8-0a2b-4b90-a52d-8886dd6c9d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:44.561584 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42623cc8-0a2b-4b90-a52d-8886dd6c9d6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:44.563889 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42623cc8-0a2b-4b90-a52d-8886dd6c9d6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:44.572220 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42623cc8-0a2b-4b90-a52d-8886dd6c9d6a admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:44.610759 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5db23e-af67-4000-a172-ff2a15435c1a admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:44.620660 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 876/1749] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:44.627081 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5db23e-af67-4000-a172-ff2a15435c1a admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:44.647361 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f64d5-ecb5-4c94-81be-2722e5e17842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:44.648669 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f64d5-ecb5-4c94-81be-2722e5e17842 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:44.649015 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f64d5-ecb5-4c94-81be-2722e5e17842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:44.649296 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f64d5-ecb5-4c94-81be-2722e5e17842 admin admin] PATH_INFO: `/v3/projects/a170f58ca748403e8f3f6636040f1a0f/users/8a2a844d5b60402e9b98343e26a1dac1/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:44.685575 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f64d5-ecb5-4c94-81be-2722e5e17842 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a170f58ca748403e8f3f6636040f1a0f, user_id=8a2a844d5b60402e9b98343e26a1dac1, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:44.693474 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f64d5-ecb5-4c94-81be-2722e5e17842 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:44.727351 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 874/1750] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:13:44 2022] PUT /identity/v3/projects/a170f58ca748403e8f3f6636040f1a0f/users/8a2a844d5b60402e9b98343e26a1dac1/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:13:44.729561 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42623cc8-0a2b-4b90-a52d-8886dd6c9d6a admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:44.747149 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42623cc8-0a2b-4b90-a52d-8886dd6c9d6a admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:44.895098 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d27942-0254-4565-bdb0-e0240d230ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:44.895737 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d27942-0254-4565-bdb0-e0240d230ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:44.896025 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d27942-0254-4565-bdb0-e0240d230ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:44.896301 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d27942-0254-4565-bdb0-e0240d230ab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:44.896960 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d27942-0254-4565-bdb0-e0240d230ab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:44.898555 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d27942-0254-4565-bdb0-e0240d230ab0 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:44.924683 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 877/1751] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:44.929709 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f64d5-ecb5-4c94-81be-2722e5e17842 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:44.947672 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f64d5-ecb5-4c94-81be-2722e5e17842 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:44.962136 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527e284-dba7-4670-9eda-5046761c2b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:44.963060 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527e284-dba7-4670-9eda-5046761c2b92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:44.963420 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527e284-dba7-4670-9eda-5046761c2b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:44.963698 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527e284-dba7-4670-9eda-5046761c2b92 admin admin] PATH_INFO: `/v3/projects/a170f58ca748403e8f3f6636040f1a0f/users/8a2a844d5b60402e9b98343e26a1dac1/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:44.966386 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527e284-dba7-4670-9eda-5046761c2b92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a170f58ca748403e8f3f6636040f1a0f, user_id=8a2a844d5b60402e9b98343e26a1dac1, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:44.971267 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527e284-dba7-4670-9eda-5046761c2b92 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:44.997711 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 875/1752] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:13:44 2022] PUT /identity/v3/projects/a170f58ca748403e8f3f6636040f1a0f/users/8a2a844d5b60402e9b98343e26a1dac1/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:45.012543 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4856fb8-8c80-4642-840c-309d1cb20954 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:45.013008 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4856fb8-8c80-4642-840c-309d1cb20954 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:45.013503 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4856fb8-8c80-4642-840c-309d1cb20954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:45.198946 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-c4856fb8-8c80-4642-840c-309d1cb20954 None None] MFA Rules not processed for user `8a2a844d5b60402e9b98343e26a1dac1`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:45.199680 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c4856fb8-8c80-4642-840c-309d1cb20954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:45.374887 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 878/1753] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:45 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:45.392474 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527e284-dba7-4670-9eda-5046761c2b92 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:45.395507 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-c527e284-dba7-4670-9eda-5046761c2b92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:45.487484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e9942-a9b6-4914-b363-323a419a3a87 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:45.487794 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e9942-a9b6-4914-b363-323a419a3a87 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:45.488075 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e9942-a9b6-4914-b363-323a419a3a87 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:45.489849 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 879/1754] 10.222.0.42 () {54 vars in 921 bytes} [Mon Sep 12 14:13:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:13:45.625397 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527e284-dba7-4670-9eda-5046761c2b92 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:45.635086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bce983-c130-48b8-bb16-ef9db1121441 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:45.635659 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bce983-c130-48b8-bb16-ef9db1121441 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:45.635659 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bce983-c130-48b8-bb16-ef9db1121441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:45.638284 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b3dee1-36ba-4ca0-9089-6bdcb675b53a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:45.641818 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b3dee1-36ba-4ca0-9089-6bdcb675b53a service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:45.642204 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b3dee1-36ba-4ca0-9089-6bdcb675b53a service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:45.642484 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b3dee1-36ba-4ca0-9089-6bdcb675b53a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:45.676045 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:45.681117 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b3dee1-36ba-4ca0-9089-6bdcb675b53a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:45.682736 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b3dee1-36ba-4ca0-9089-6bdcb675b53a service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:45.695550 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-e3bce983-c130-48b8-bb16-ef9db1121441 None None] MFA Rules not processed for user `854ab2dcec5a48b18f409b33cb61b5da`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:45.696815 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e3bce983-c130-48b8-bb16-ef9db1121441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:45.718093 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 876/1755] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:45 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:13:45.872976 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b3dee1-36ba-4ca0-9089-6bdcb675b53a service cinder] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:45.904681 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 880/1756] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Sep 12 14:13:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:46.045315 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b3dee1-36ba-4ca0-9089-6bdcb675b53a service cinder] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:46.067753 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac45246-8ff8-45e1-8f51-fee108733bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:46.069099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac45246-8ff8-45e1-8f51-fee108733bc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:46.069383 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac45246-8ff8-45e1-8f51-fee108733bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:46.069681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac45246-8ff8-45e1-8f51-fee108733bc2 admin admin] PATH_INFO: `/v3/users/cdab6cc4ba5d4a3b999d28252411c6fd` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:46.075975 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac45246-8ff8-45e1-8f51-fee108733bc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdab6cc4ba5d4a3b999d28252411c6fd)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:46.083836 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac45246-8ff8-45e1-8f51-fee108733bc2 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:46.217350 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:46.251630 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-fac45246-8ff8-45e1-8f51-fee108733bc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdab6cc4ba5d4a3b999d28252411c6fd'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:46.271536 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-fac45246-8ff8-45e1-8f51-fee108733bc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdab6cc4ba5d4a3b999d28252411c6fd'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:46.272239 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-fac45246-8ff8-45e1-8f51-fee108733bc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdab6cc4ba5d4a3b999d28252411c6fd'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:46.273181 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-fac45246-8ff8-45e1-8f51-fee108733bc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdab6cc4ba5d4a3b999d28252411c6fd'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:46.293967 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-fac45246-8ff8-45e1-8f51-fee108733bc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdab6cc4ba5d4a3b999d28252411c6fd'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:46.339892 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 877/1757] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:13:45 2022] DELETE /identity/v3/users/cdab6cc4ba5d4a3b999d28252411c6fd => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:46.344406 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb07b22-6b7b-40ea-9893-52395b86dded None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:46.345622 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb07b22-6b7b-40ea-9893-52395b86dded None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:46.346187 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb07b22-6b7b-40ea-9893-52395b86dded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:46.382612 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-ccb07b22-6b7b-40ea-9893-52395b86dded None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:46.384053 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ccb07b22-6b7b-40ea-9893-52395b86dded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:46.529060 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 881/1758] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:46.551758 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac45246-8ff8-45e1-8f51-fee108733bc2 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:46.553635 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-fac45246-8ff8-45e1-8f51-fee108733bc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:46.632524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb07b22-6b7b-40ea-9893-52395b86dded None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:46.636987 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ccb07b22-6b7b-40ea-9893-52395b86dded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:46.800047 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac45246-8ff8-45e1-8f51-fee108733bc2 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:46.805589 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e486cf4-0f28-4256-8ab6-50e4597a5692 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:46.805765 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e486cf4-0f28-4256-8ab6-50e4597a5692 service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:46.806101 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e486cf4-0f28-4256-8ab6-50e4597a5692 service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:46.807482 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e486cf4-0f28-4256-8ab6-50e4597a5692 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:46.820707 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e486cf4-0f28-4256-8ab6-50e4597a5692 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:46.822559 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e486cf4-0f28-4256-8ab6-50e4597a5692 service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:46.840558 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 878/1759] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:13:46.862787 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e486cf4-0f28-4256-8ab6-50e4597a5692 service neutron] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:46.867439 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-7e486cf4-0f28-4256-8ab6-50e4597a5692 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:46.928173 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e486cf4-0f28-4256-8ab6-50e4597a5692 service neutron] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:46.955097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb07b22-6b7b-40ea-9893-52395b86dded None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:46.969755 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d546d-6ea7-4e4a-b8df-6ebb8229fe86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:46.970462 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d546d-6ea7-4e4a-b8df-6ebb8229fe86 service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:46.970737 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d546d-6ea7-4e4a-b8df-6ebb8229fe86 service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:46.971087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d546d-6ea7-4e4a-b8df-6ebb8229fe86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:46.975906 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-147d546d-6ea7-4e4a-b8df-6ebb8229fe86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:46.982510 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89db1c9-291b-479a-9e69-40d951c3b5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:46.984900 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89db1c9-291b-479a-9e69-40d951c3b5eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:46.992427 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89db1c9-291b-479a-9e69-40d951c3b5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:46.993721 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89db1c9-291b-479a-9e69-40d951c3b5eb admin admin] PATH_INFO: `/v3/projects/8c01e89f44654a9dba00b4858a03fd8f` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:46.999013 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89db1c9-291b-479a-9e69-40d951c3b5eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c01e89f44654a9dba00b4858a03fd8f)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:47.006526 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d546d-6ea7-4e4a-b8df-6ebb8229fe86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:47.008188 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89db1c9-291b-479a-9e69-40d951c3b5eb admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:47.008397 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d546d-6ea7-4e4a-b8df-6ebb8229fe86 service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:47.030100 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 882/1760] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:13:47.183915 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-d89db1c9-291b-479a-9e69-40d951c3b5eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c01e89f44654a9dba00b4858a03fd8f', 'request_id': 'req-d89db1c9-291b-479a-9e69-40d951c3b5eb'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:47.207569 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-d89db1c9-291b-479a-9e69-40d951c3b5eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c01e89f44654a9dba00b4858a03fd8f', 'request_id': 'req-d89db1c9-291b-479a-9e69-40d951c3b5eb'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:47.233214 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-d89db1c9-291b-479a-9e69-40d951c3b5eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c01e89f44654a9dba00b4858a03fd8f', 'request_id': 'req-d89db1c9-291b-479a-9e69-40d951c3b5eb'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:47.277868 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-d89db1c9-291b-479a-9e69-40d951c3b5eb admin admin] The token cache is being invalidate because project 8c01e89f44654a9dba00b4858a03fd8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:13:47.278333 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-d89db1c9-291b-479a-9e69-40d951c3b5eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:47.279482 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-d89db1c9-291b-479a-9e69-40d951c3b5eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:47.281236 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 879/1761] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:13:46 2022] DELETE /identity/v3/projects/8c01e89f44654a9dba00b4858a03fd8f => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:47.287036 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad653e9-88a7-414b-8f4c-8de75f5da57e None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:47.287354 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad653e9-88a7-414b-8f4c-8de75f5da57e None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:47.287635 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad653e9-88a7-414b-8f4c-8de75f5da57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:47.348364 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-cad653e9-88a7-414b-8f4c-8de75f5da57e None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:47.349816 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-cad653e9-88a7-414b-8f4c-8de75f5da57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:47.449208 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:47.599289 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 883/1762] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:47.608004 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83fe0e4-0868-4734-a707-505d22f6ed47 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:47.609080 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83fe0e4-0868-4734-a707-505d22f6ed47 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:47.609786 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83fe0e4-0868-4734-a707-505d22f6ed47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:47.668630 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-e83fe0e4-0868-4734-a707-505d22f6ed47 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:47.672211 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-e83fe0e4-0868-4734-a707-505d22f6ed47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:47.704768 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 880/1763] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:47.709970 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad653e9-88a7-414b-8f4c-8de75f5da57e None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:47.728137 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad653e9-88a7-414b-8f4c-8de75f5da57e None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:47.745632 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e5b58c-9695-40d8-9b2e-5d464d219811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:47.746221 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5b58c-9695-40d8-9b2e-5d464d219811 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:47.746486 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5b58c-9695-40d8-9b2e-5d464d219811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:47.746814 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5b58c-9695-40d8-9b2e-5d464d219811 admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:47.747546 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e5b58c-9695-40d8-9b2e-5d464d219811 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:13:47.747875 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e5b58c-9695-40d8-9b2e-5d464d219811 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:47.749451 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e5b58c-9695-40d8-9b2e-5d464d219811 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:47.788996 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 884/1764] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:13:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:13:47.790856 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83fe0e4-0868-4734-a707-505d22f6ed47 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:47.822614 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83fe0e4-0868-4734-a707-505d22f6ed47 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:47.838071 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4610289-24eb-470b-90da-60d0375603b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:47.838622 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4610289-24eb-470b-90da-60d0375603b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:47.838977 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4610289-24eb-470b-90da-60d0375603b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:47.839259 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4610289-24eb-470b-90da-60d0375603b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:47.847596 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4610289-24eb-470b-90da-60d0375603b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:47.851629 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4610289-24eb-470b-90da-60d0375603b6 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:47.965455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 881/1765] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:13:47 2022] POST /identity/v3/projects => generated 369 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:47.969839 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e5b58c-9695-40d8-9b2e-5d464d219811 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:47.991835 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e5b58c-9695-40d8-9b2e-5d464d219811 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:48.180905 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f335c-bbe4-456e-93bb-266a6539b682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:48.181615 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f335c-bbe4-456e-93bb-266a6539b682 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:48.181807 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f335c-bbe4-456e-93bb-266a6539b682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:48.182088 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f335c-bbe4-456e-93bb-266a6539b682 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:48.182735 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68f335c-bbe4-456e-93bb-266a6539b682 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:48.185226 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68f335c-bbe4-456e-93bb-266a6539b682 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:48.237721 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 885/1766] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:13:47 2022] POST /identity/v3/users => generated 285 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:48.243146 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4610289-24eb-470b-90da-60d0375603b6 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:48.263829 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4610289-24eb-470b-90da-60d0375603b6 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:48.289689 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0bd27e-cf5d-4c82-888f-44a45d1e09fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:48.290270 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0bd27e-cf5d-4c82-888f-44a45d1e09fe admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:48.290628 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0bd27e-cf5d-4c82-888f-44a45d1e09fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:48.290815 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0bd27e-cf5d-4c82-888f-44a45d1e09fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:48.296629 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0bd27e-cf5d-4c82-888f-44a45d1e09fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:48.298867 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0bd27e-cf5d-4c82-888f-44a45d1e09fe admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:48.332692 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:48.363558 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 882/1767] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:48.365486 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f335c-bbe4-456e-93bb-266a6539b682 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:48.380136 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f335c-bbe4-456e-93bb-266a6539b682 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:48.392206 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98538e7c-07fc-4797-aaaf-267ed33c0c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:48.393058 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98538e7c-07fc-4797-aaaf-267ed33c0c71 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:48.393349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98538e7c-07fc-4797-aaaf-267ed33c0c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:48.394136 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98538e7c-07fc-4797-aaaf-267ed33c0c71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:48.397344 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98538e7c-07fc-4797-aaaf-267ed33c0c71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:48.399107 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98538e7c-07fc-4797-aaaf-267ed33c0c71 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:48.435630 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 886/1768] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:48.440616 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0bd27e-cf5d-4c82-888f-44a45d1e09fe admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:48.454335 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0bd27e-cf5d-4c82-888f-44a45d1e09fe admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:48.460893 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf88ea18-e901-4c6f-af92-9176c64e3838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:48.462068 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf88ea18-e901-4c6f-af92-9176c64e3838 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:48.462469 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf88ea18-e901-4c6f-af92-9176c64e3838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:48.462987 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf88ea18-e901-4c6f-af92-9176c64e3838 admin admin] PATH_INFO: `/v3/projects/30b7cb46a2234ea9b928a3fc48abddfa/users/d367dcbee8fa49feb6dc0a6a07f85dfd/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:48.486345 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf88ea18-e901-4c6f-af92-9176c64e3838 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30b7cb46a2234ea9b928a3fc48abddfa, user_id=d367dcbee8fa49feb6dc0a6a07f85dfd, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:48.490224 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf88ea18-e901-4c6f-af92-9176c64e3838 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:48.518654 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 883/1769] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:13:48 2022] PUT /identity/v3/projects/30b7cb46a2234ea9b928a3fc48abddfa/users/d367dcbee8fa49feb6dc0a6a07f85dfd/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:48.541371 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6581107a-6d58-4e83-a86a-33c42bfcf49a None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:48.541747 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6581107a-6d58-4e83-a86a-33c42bfcf49a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:48.541965 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6581107a-6d58-4e83-a86a-33c42bfcf49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:48.619956 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-6581107a-6d58-4e83-a86a-33c42bfcf49a None None] MFA Rules not processed for user `d367dcbee8fa49feb6dc0a6a07f85dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:48.622397 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-6581107a-6d58-4e83-a86a-33c42bfcf49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:48.807078 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf88ea18-e901-4c6f-af92-9176c64e3838 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:48.813471 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 887/1770] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:49.001594 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf88ea18-e901-4c6f-af92-9176c64e3838 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:49.019810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de69e5-9a7a-4a4e-a1c4-12b97e2993b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:49.021317 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de69e5-9a7a-4a4e-a1c4-12b97e2993b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:49.021700 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de69e5-9a7a-4a4e-a1c4-12b97e2993b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:49.022137 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de69e5-9a7a-4a4e-a1c4-12b97e2993b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:49.023397 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de69e5-9a7a-4a4e-a1c4-12b97e2993b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:49.025577 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de69e5-9a7a-4a4e-a1c4-12b97e2993b6 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:49.058148 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6581107a-6d58-4e83-a86a-33c42bfcf49a None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:49.060104 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-6581107a-6d58-4e83-a86a-33c42bfcf49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:49.094934 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6581107a-6d58-4e83-a86a-33c42bfcf49a None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:49.116251 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 884/1771] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:13:48 2022] POST /identity/v3/projects => generated 369 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:49.117285 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eabf02-bc75-4138-92af-f4a4c03fe479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:49.117727 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de69e5-9a7a-4a4e-a1c4-12b97e2993b6 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:49.119943 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eabf02-bc75-4138-92af-f4a4c03fe479 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:49.120298 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eabf02-bc75-4138-92af-f4a4c03fe479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:49.123456 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eabf02-bc75-4138-92af-f4a4c03fe479 admin admin] PATH_INFO: `/v3/users/a4d1432f4d5349048189f1017199691e` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:49.125048 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eabf02-bc75-4138-92af-f4a4c03fe479 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4d1432f4d5349048189f1017199691e)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:49.130559 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eabf02-bc75-4138-92af-f4a4c03fe479 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:49.186153 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:49.303707 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-65eabf02-bc75-4138-92af-f4a4c03fe479 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4d1432f4d5349048189f1017199691e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:49.322874 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-65eabf02-bc75-4138-92af-f4a4c03fe479 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4d1432f4d5349048189f1017199691e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:49.323841 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-65eabf02-bc75-4138-92af-f4a4c03fe479 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4d1432f4d5349048189f1017199691e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:49.324480 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-65eabf02-bc75-4138-92af-f4a4c03fe479 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4d1432f4d5349048189f1017199691e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:49.364433 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-65eabf02-bc75-4138-92af-f4a4c03fe479 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4d1432f4d5349048189f1017199691e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:49.374003 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de69e5-9a7a-4a4e-a1c4-12b97e2993b6 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:49.388544 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3188102-89e9-48a4-b681-98a28c33d805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:49.389737 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3188102-89e9-48a4-b681-98a28c33d805 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:49.389737 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3188102-89e9-48a4-b681-98a28c33d805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:49.389737 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3188102-89e9-48a4-b681-98a28c33d805 admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:49.395137 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3188102-89e9-48a4-b681-98a28c33d805 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:49.398938 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3188102-89e9-48a4-b681-98a28c33d805 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:49.409411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 888/1772] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:13:49 2022] DELETE /identity/v3/users/a4d1432f4d5349048189f1017199691e => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:49.410510 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eabf02-bc75-4138-92af-f4a4c03fe479 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:49.415158 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-65eabf02-bc75-4138-92af-f4a4c03fe479 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:49.478211 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 885/1773] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:13:49 2022] POST /identity/v3/users => generated 291 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:13:49.479908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52546919-b72a-4eec-891a-732c3d597533 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:49.485470 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52546919-b72a-4eec-891a-732c3d597533 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:49.487176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52546919-b72a-4eec-891a-732c3d597533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:49.554943 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-52546919-b72a-4eec-891a-732c3d597533 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:49.556272 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-52546919-b72a-4eec-891a-732c3d597533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:49.587718 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eabf02-bc75-4138-92af-f4a4c03fe479 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:49.593442 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c5934-1c17-4290-a38d-0ad83849226b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:49.594363 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c5934-1c17-4290-a38d-0ad83849226b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:49.594811 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c5934-1c17-4290-a38d-0ad83849226b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:49.595916 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c5934-1c17-4290-a38d-0ad83849226b admin admin] PATH_INFO: `/v3/projects/767d656ee1ef48ed83675e3f0e1769f1` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:49.597283 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c5934-1c17-4290-a38d-0ad83849226b admin admin] RBAC: Authorizing `identity:delete_project(project_id=767d656ee1ef48ed83675e3f0e1769f1)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:49.599473 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c5934-1c17-4290-a38d-0ad83849226b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:49.675127 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:49.708183 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-486c5934-1c17-4290-a38d-0ad83849226b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '767d656ee1ef48ed83675e3f0e1769f1', 'request_id': 'req-486c5934-1c17-4290-a38d-0ad83849226b'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:49.711533 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 886/1774] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:49.712236 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52546919-b72a-4eec-891a-732c3d597533 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:49.721313 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-52546919-b72a-4eec-891a-732c3d597533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:49.739102 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52546919-b72a-4eec-891a-732c3d597533 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:49.758392 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-486c5934-1c17-4290-a38d-0ad83849226b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '767d656ee1ef48ed83675e3f0e1769f1', 'request_id': 'req-486c5934-1c17-4290-a38d-0ad83849226b'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:49.795162 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-486c5934-1c17-4290-a38d-0ad83849226b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '767d656ee1ef48ed83675e3f0e1769f1', 'request_id': 'req-486c5934-1c17-4290-a38d-0ad83849226b'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:49.877507 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-486c5934-1c17-4290-a38d-0ad83849226b admin admin] The token cache is being invalidate because project 767d656ee1ef48ed83675e3f0e1769f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:13:49.879667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-486c5934-1c17-4290-a38d-0ad83849226b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:49.884201 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-486c5934-1c17-4290-a38d-0ad83849226b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:49.889648 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 889/1775] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:13:49 2022] DELETE /identity/v3/projects/767d656ee1ef48ed83675e3f0e1769f1 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:49.893355 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06736885-f0b8-4fc7-bc89-62c77b4b835a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:49.894006 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06736885-f0b8-4fc7-bc89-62c77b4b835a admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:49.894297 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06736885-f0b8-4fc7-bc89-62c77b4b835a admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:49.894571 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06736885-f0b8-4fc7-bc89-62c77b4b835a admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:49.895002 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c5934-1c17-4290-a38d-0ad83849226b admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:49.895307 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06736885-f0b8-4fc7-bc89-62c77b4b835a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:49.901477 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-486c5934-1c17-4290-a38d-0ad83849226b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:49.906053 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06736885-f0b8-4fc7-bc89-62c77b4b835a admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:49.961109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 887/1776] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:49.962065 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06736885-f0b8-4fc7-bc89-62c77b4b835a admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:49.983471 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-06736885-f0b8-4fc7-bc89-62c77b4b835a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:50.018047 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06736885-f0b8-4fc7-bc89-62c77b4b835a admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:50.040519 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81a3e5-07af-4ba0-aecd-0b4e90fa0159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:50.041207 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81a3e5-07af-4ba0-aecd-0b4e90fa0159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:50.041495 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81a3e5-07af-4ba0-aecd-0b4e90fa0159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:50.041765 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81a3e5-07af-4ba0-aecd-0b4e90fa0159 admin admin] PATH_INFO: `/v3/users/8a2a844d5b60402e9b98343e26a1dac1` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:50.045555 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc81a3e5-07af-4ba0-aecd-0b4e90fa0159 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a2a844d5b60402e9b98343e26a1dac1)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:50.048136 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc81a3e5-07af-4ba0-aecd-0b4e90fa0159 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:50.055311 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:50.178323 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-fc81a3e5-07af-4ba0-aecd-0b4e90fa0159 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a2a844d5b60402e9b98343e26a1dac1'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:50.190607 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-fc81a3e5-07af-4ba0-aecd-0b4e90fa0159 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a2a844d5b60402e9b98343e26a1dac1'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:50.191286 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-fc81a3e5-07af-4ba0-aecd-0b4e90fa0159 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a2a844d5b60402e9b98343e26a1dac1'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:50.191925 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-fc81a3e5-07af-4ba0-aecd-0b4e90fa0159 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a2a844d5b60402e9b98343e26a1dac1'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:50.202143 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c5934-1c17-4290-a38d-0ad83849226b admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:50.213640 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-fc81a3e5-07af-4ba0-aecd-0b4e90fa0159 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a2a844d5b60402e9b98343e26a1dac1'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:50.221919 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d01e8-5a50-45b1-bc80-c618040fd76b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:50.224643 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d01e8-5a50-45b1-bc80-c618040fd76b service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:50.224948 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d01e8-5a50-45b1-bc80-c618040fd76b service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:50.225225 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d01e8-5a50-45b1-bc80-c618040fd76b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:50.241996 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-bf7d01e8-5a50-45b1-bc80-c618040fd76b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:50.257645 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 888/1777] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:13:49 2022] DELETE /identity/v3/users/8a2a844d5b60402e9b98343e26a1dac1 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:50.258732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81a3e5-07af-4ba0-aecd-0b4e90fa0159 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:50.261557 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-fc81a3e5-07af-4ba0-aecd-0b4e90fa0159 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:50.295574 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7d01e8-5a50-45b1-bc80-c618040fd76b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:50.300811 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7d01e8-5a50-45b1-bc80-c618040fd76b service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:50.332592 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 890/1778] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:13:50.443741 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81a3e5-07af-4ba0-aecd-0b4e90fa0159 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:50.459226 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a069efba-d2be-4f12-8778-b3af9d862b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:50.461457 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a069efba-d2be-4f12-8778-b3af9d862b1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:50.463198 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a069efba-d2be-4f12-8778-b3af9d862b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:50.463656 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a069efba-d2be-4f12-8778-b3af9d862b1e admin admin] PATH_INFO: `/v3/projects/dde71a33c3e54e14812725ef913bf2e1/users/8d8e02415e854b4a92bd457dba02e1f9/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:50.467208 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:50.492880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a069efba-d2be-4f12-8778-b3af9d862b1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dde71a33c3e54e14812725ef913bf2e1, user_id=8d8e02415e854b4a92bd457dba02e1f9, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:50.495805 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a069efba-d2be-4f12-8778-b3af9d862b1e admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:50.531217 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d01e8-5a50-45b1-bc80-c618040fd76b service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:50.541249 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 889/1779] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:13:50 2022] PUT /identity/v3/projects/dde71a33c3e54e14812725ef913bf2e1/users/8d8e02415e854b4a92bd457dba02e1f9/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:50.554963 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d01e8-5a50-45b1-bc80-c618040fd76b service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:50.692148 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a198f084-0eab-4686-9779-60a6500a60f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:50.694602 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198f084-0eab-4686-9779-60a6500a60f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:50.695028 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198f084-0eab-4686-9779-60a6500a60f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:50.696180 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198f084-0eab-4686-9779-60a6500a60f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:50.697291 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a198f084-0eab-4686-9779-60a6500a60f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:50.703516 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a198f084-0eab-4686-9779-60a6500a60f0 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:50.729792 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 891/1780] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:50.733035 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a069efba-d2be-4f12-8778-b3af9d862b1e admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:50.746570 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a069efba-d2be-4f12-8778-b3af9d862b1e admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:50.773823 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5880c1-54cb-42b4-af09-e6c0fb44e7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:50.774566 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5880c1-54cb-42b4-af09-e6c0fb44e7aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:50.774860 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5880c1-54cb-42b4-af09-e6c0fb44e7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:50.775129 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5880c1-54cb-42b4-af09-e6c0fb44e7aa admin admin] PATH_INFO: `/v3/projects/dde71a33c3e54e14812725ef913bf2e1/users/8d8e02415e854b4a92bd457dba02e1f9/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:50.783831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5880c1-54cb-42b4-af09-e6c0fb44e7aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=dde71a33c3e54e14812725ef913bf2e1, user_id=8d8e02415e854b4a92bd457dba02e1f9, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:50.791672 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5880c1-54cb-42b4-af09-e6c0fb44e7aa admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:50.812108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236fe2d9-09fc-4815-a359-c0542ca25300 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:50.812355 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236fe2d9-09fc-4815-a359-c0542ca25300 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:50.812820 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236fe2d9-09fc-4815-a359-c0542ca25300 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:50.816632 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 892/1781] 10.222.0.42 () {54 vars in 921 bytes} [Mon Sep 12 14:13:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:13:50.841903 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 890/1782] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:13:50 2022] PUT /identity/v3/projects/dde71a33c3e54e14812725ef913bf2e1/users/8d8e02415e854b4a92bd457dba02e1f9/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:50.856647 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f805f7-41e9-4afb-bc8c-39d96271d1b0 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:50.857523 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f805f7-41e9-4afb-bc8c-39d96271d1b0 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:50.858006 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f805f7-41e9-4afb-bc8c-39d96271d1b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:50.947097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-84f805f7-41e9-4afb-bc8c-39d96271d1b0 None None] MFA Rules not processed for user `8d8e02415e854b4a92bd457dba02e1f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:50.952563 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-84f805f7-41e9-4afb-bc8c-39d96271d1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:51.149261 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 893/1783] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:50 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:51.162763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5880c1-54cb-42b4-af09-e6c0fb44e7aa admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:51.164657 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-ad5880c1-54cb-42b4-af09-e6c0fb44e7aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:51.289735 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5880c1-54cb-42b4-af09-e6c0fb44e7aa admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:51.299965 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca45baf-34ce-4449-bf76-2e666fde5da1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:51.300785 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca45baf-34ce-4449-bf76-2e666fde5da1 service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:51.301069 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca45baf-34ce-4449-bf76-2e666fde5da1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:51.301344 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca45baf-34ce-4449-bf76-2e666fde5da1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:51.321534 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca45baf-34ce-4449-bf76-2e666fde5da1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:51.329165 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca45baf-34ce-4449-bf76-2e666fde5da1 service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:51.356677 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 891/1784] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:51 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:13:51.370193 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c2050-a014-45f1-9445-4bd96ad10fc4 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:51.370824 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c2050-a014-45f1-9445-4bd96ad10fc4 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:51.371273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c2050-a014-45f1-9445-4bd96ad10fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:51.422955 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-928c2050-a014-45f1-9445-4bd96ad10fc4 None None] MFA Rules not processed for user `854ab2dcec5a48b18f409b33cb61b5da`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:51.424585 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-928c2050-a014-45f1-9445-4bd96ad10fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:51.507792 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:51.544372 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 894/1785] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Sep 12 14:13:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:51.978221 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca45baf-34ce-4449-bf76-2e666fde5da1 service cinder] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:51.978221 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-0ca45baf-34ce-4449-bf76-2e666fde5da1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:52.006985 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca45baf-34ce-4449-bf76-2e666fde5da1 service cinder] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:52.013651 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d042cd-48e5-4f31-a040-25a110cafd87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:52.014273 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d042cd-48e5-4f31-a040-25a110cafd87 service nova] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:52.014555 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d042cd-48e5-4f31-a040-25a110cafd87 service nova] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:52.014829 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d042cd-48e5-4f31-a040-25a110cafd87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:52.033860 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d042cd-48e5-4f31-a040-25a110cafd87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:52.035825 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d042cd-48e5-4f31-a040-25a110cafd87 service nova] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:52.064847 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 892/1786] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:13:52.512070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c2050-a014-45f1-9445-4bd96ad10fc4 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:52.513568 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-928c2050-a014-45f1-9445-4bd96ad10fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:52.669497 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d042cd-48e5-4f31-a040-25a110cafd87 service nova] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:52.671788 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-35d042cd-48e5-4f31-a040-25a110cafd87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:52.742366 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c2050-a014-45f1-9445-4bd96ad10fc4 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:52.756162 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4967d7-6a14-41ea-854b-72040a92fbea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:52.756891 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4967d7-6a14-41ea-854b-72040a92fbea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:52.757221 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4967d7-6a14-41ea-854b-72040a92fbea admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:52.757576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4967d7-6a14-41ea-854b-72040a92fbea admin admin] PATH_INFO: `/v3/projects/a170f58ca748403e8f3f6636040f1a0f` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:52.758908 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4967d7-6a14-41ea-854b-72040a92fbea admin admin] RBAC: Authorizing `identity:delete_project(project_id=a170f58ca748403e8f3f6636040f1a0f)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:52.761162 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4967d7-6a14-41ea-854b-72040a92fbea admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:52.792159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d042cd-48e5-4f31-a040-25a110cafd87 service nova] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:52.799318 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419f8b5f-47e6-4469-bd98-a2fd9daae2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:52.800787 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f8b5f-47e6-4469-bd98-a2fd9daae2c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:52.801249 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f8b5f-47e6-4469-bd98-a2fd9daae2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:52.801689 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f8b5f-47e6-4469-bd98-a2fd9daae2c0 admin admin] PATH_INFO: `/v3/projects/7e3af2264bad4107928e28327806d8a8` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:52.803384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419f8b5f-47e6-4469-bd98-a2fd9daae2c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e3af2264bad4107928e28327806d8a8)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:52.805309 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419f8b5f-47e6-4469-bd98-a2fd9daae2c0 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:52.857792 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-6a4967d7-6a14-41ea-854b-72040a92fbea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a170f58ca748403e8f3f6636040f1a0f', 'request_id': 'req-6a4967d7-6a14-41ea-854b-72040a92fbea'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:52.895393 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-6a4967d7-6a14-41ea-854b-72040a92fbea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a170f58ca748403e8f3f6636040f1a0f', 'request_id': 'req-6a4967d7-6a14-41ea-854b-72040a92fbea'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:52.911542 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-6a4967d7-6a14-41ea-854b-72040a92fbea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a170f58ca748403e8f3f6636040f1a0f', 'request_id': 'req-6a4967d7-6a14-41ea-854b-72040a92fbea'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:52.929646 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-6a4967d7-6a14-41ea-854b-72040a92fbea admin admin] The token cache is being invalidate because project a170f58ca748403e8f3f6636040f1a0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:13:52.931092 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-6a4967d7-6a14-41ea-854b-72040a92fbea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:52.931842 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-6a4967d7-6a14-41ea-854b-72040a92fbea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:52.941276 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 895/1787] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:13:52 2022] DELETE /identity/v3/projects/a170f58ca748403e8f3f6636040f1a0f => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:52.944123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30f8d8-99ef-4f68-ae73-3d409111c33d None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:52.944983 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-419f8b5f-47e6-4469-bd98-a2fd9daae2c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e3af2264bad4107928e28327806d8a8', 'request_id': 'req-419f8b5f-47e6-4469-bd98-a2fd9daae2c0'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:52.947286 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30f8d8-99ef-4f68-ae73-3d409111c33d None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:52.948063 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30f8d8-99ef-4f68-ae73-3d409111c33d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:52.958639 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-419f8b5f-47e6-4469-bd98-a2fd9daae2c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e3af2264bad4107928e28327806d8a8', 'request_id': 'req-419f8b5f-47e6-4469-bd98-a2fd9daae2c0'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:52.961627 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:52.972798 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-419f8b5f-47e6-4469-bd98-a2fd9daae2c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e3af2264bad4107928e28327806d8a8', 'request_id': 'req-419f8b5f-47e6-4469-bd98-a2fd9daae2c0'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:52.998725 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-419f8b5f-47e6-4469-bd98-a2fd9daae2c0 admin admin] The token cache is being invalidate because project 7e3af2264bad4107928e28327806d8a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:13:52.999141 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-419f8b5f-47e6-4469-bd98-a2fd9daae2c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:52.999881 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-419f8b5f-47e6-4469-bd98-a2fd9daae2c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:53.002463 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-ce30f8d8-99ef-4f68-ae73-3d409111c33d None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:53.005541 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ce30f8d8-99ef-4f68-ae73-3d409111c33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:53.006454 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 893/1788] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:13:52 2022] DELETE /identity/v3/projects/7e3af2264bad4107928e28327806d8a8 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:53.021176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd410a-8fa9-475e-ad38-17dee425fb0f None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:53.021468 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd410a-8fa9-475e-ad38-17dee425fb0f None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:53.021744 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd410a-8fa9-475e-ad38-17dee425fb0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:53.057651 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-6bbd410a-8fa9-475e-ad38-17dee425fb0f None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:53.058971 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-6bbd410a-8fa9-475e-ad38-17dee425fb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:53.143891 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 896/1789] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:53.151406 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf354f9a-fc0d-47bf-b93b-29819f292579 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:53.151851 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf354f9a-fc0d-47bf-b93b-29819f292579 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:53.152140 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf354f9a-fc0d-47bf-b93b-29819f292579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:53.195271 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-bf354f9a-fc0d-47bf-b93b-29819f292579 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:53.196641 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-bf354f9a-fc0d-47bf-b93b-29819f292579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:53.222616 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 897/1790] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:53.223825 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf354f9a-fc0d-47bf-b93b-29819f292579 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:53.232334 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 894/1791] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:53.233651 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:53.234404 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014d0b0-60f4-47bf-b41f-7e503f2d6313 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:53.234811 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014d0b0-60f4-47bf-b41f-7e503f2d6313 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:53.235221 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014d0b0-60f4-47bf-b41f-7e503f2d6313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:53.241617 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf354f9a-fc0d-47bf-b93b-29819f292579 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:53.251294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586a3e6-7e23-4f7c-97bb-05cc0c351c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:53.252417 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586a3e6-7e23-4f7c-97bb-05cc0c351c36 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:53.252923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586a3e6-7e23-4f7c-97bb-05cc0c351c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:53.253415 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586a3e6-7e23-4f7c-97bb-05cc0c351c36 admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:53.254366 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586a3e6-7e23-4f7c-97bb-05cc0c351c36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:13:53.256030 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586a3e6-7e23-4f7c-97bb-05cc0c351c36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:53.258306 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586a3e6-7e23-4f7c-97bb-05cc0c351c36 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:53.281328 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 898/1792] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:13:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:13:53.282907 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586a3e6-7e23-4f7c-97bb-05cc0c351c36 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:53.285778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-1014d0b0-60f4-47bf-b41f-7e503f2d6313 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:53.287288 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-1014d0b0-60f4-47bf-b41f-7e503f2d6313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:53.296181 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 895/1793] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:13:53.300920 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014d0b0-60f4-47bf-b41f-7e503f2d6313 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:53.309120 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586a3e6-7e23-4f7c-97bb-05cc0c351c36 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:53.318223 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3463cc3c-4444-4e0c-bacc-9724731a5dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:53.319239 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3463cc3c-4444-4e0c-bacc-9724731a5dc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:53.319677 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3463cc3c-4444-4e0c-bacc-9724731a5dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:53.320278 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3463cc3c-4444-4e0c-bacc-9724731a5dc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:53.321082 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3463cc3c-4444-4e0c-bacc-9724731a5dc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:53.323302 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3463cc3c-4444-4e0c-bacc-9724731a5dc2 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:53.342539 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014d0b0-60f4-47bf-b41f-7e503f2d6313 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:53.366396 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee012a6-d388-4e10-b54f-34293bfc6f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:53.368341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee012a6-d388-4e10-b54f-34293bfc6f39 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:53.369327 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee012a6-d388-4e10-b54f-34293bfc6f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:53.369749 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee012a6-d388-4e10-b54f-34293bfc6f39 admin admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:53.372976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 899/1794] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:13:53 2022] POST /identity/v3/projects => generated 401 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:53.373295 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee012a6-d388-4e10-b54f-34293bfc6f39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:13:53.373773 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee012a6-d388-4e10-b54f-34293bfc6f39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:53.375709 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee012a6-d388-4e10-b54f-34293bfc6f39 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:53.377296 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3463cc3c-4444-4e0c-bacc-9724731a5dc2 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:53.404112 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 896/1795] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:13:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:13:53.409149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee012a6-d388-4e10-b54f-34293bfc6f39 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:53.421907 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee012a6-d388-4e10-b54f-34293bfc6f39 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:53.482523 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3463cc3c-4444-4e0c-bacc-9724731a5dc2 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:53.490717 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70060e95-2c80-434f-aad5-01e1a4b4601f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:53.492836 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70060e95-2c80-434f-aad5-01e1a4b4601f admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:53.493273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70060e95-2c80-434f-aad5-01e1a4b4601f admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:53.493637 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70060e95-2c80-434f-aad5-01e1a4b4601f admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:53.494383 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70060e95-2c80-434f-aad5-01e1a4b4601f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:53.519716 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e6112-9e44-4a26-be08-daafe4ec51a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:53.520462 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e6112-9e44-4a26-be08-daafe4ec51a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:53.520828 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e6112-9e44-4a26-be08-daafe4ec51a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:53.521230 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e6112-9e44-4a26-be08-daafe4ec51a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:53.522033 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e6112-9e44-4a26-be08-daafe4ec51a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:53.524158 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e6112-9e44-4a26-be08-daafe4ec51a1 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:53.524732 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70060e95-2c80-434f-aad5-01e1a4b4601f admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:53.552491 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 900/1796] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:13:53 2022] POST /identity/v3/users => generated 301 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:53.557058 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70060e95-2c80-434f-aad5-01e1a4b4601f admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:53.570489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70060e95-2c80-434f-aad5-01e1a4b4601f admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:53.580468 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 897/1797] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:13:53 2022] POST /identity/v3/projects => generated 383 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:13:53.582280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e6112-9e44-4a26-be08-daafe4ec51a1 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:53.583298 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003cacf-fcb0-4e5d-be9b-21e90f959d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:53.584646 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003cacf-fcb0-4e5d-be9b-21e90f959d03 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:53.585191 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003cacf-fcb0-4e5d-be9b-21e90f959d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:53.585609 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003cacf-fcb0-4e5d-be9b-21e90f959d03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:53.586431 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b003cacf-fcb0-4e5d-be9b-21e90f959d03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:53.588465 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b003cacf-fcb0-4e5d-be9b-21e90f959d03 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:53.603349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 901/1798] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:53.607375 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003cacf-fcb0-4e5d-be9b-21e90f959d03 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:53.619156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003cacf-fcb0-4e5d-be9b-21e90f959d03 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:53.680266 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:53.686030 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e6112-9e44-4a26-be08-daafe4ec51a1 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:53.693155 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc67700-360c-4d00-9baf-e1e8f16ea6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:53.694294 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc67700-360c-4d00-9baf-e1e8f16ea6f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:53.694681 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652926ed-acbf-4dfa-ae54-e2d6b705f1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:53.694823 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc67700-360c-4d00-9baf-e1e8f16ea6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:53.695206 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc67700-360c-4d00-9baf-e1e8f16ea6f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:53.695834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652926ed-acbf-4dfa-ae54-e2d6b705f1e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:53.696125 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652926ed-acbf-4dfa-ae54-e2d6b705f1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:53.696316 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc67700-360c-4d00-9baf-e1e8f16ea6f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:53.696906 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652926ed-acbf-4dfa-ae54-e2d6b705f1e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:53.697657 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652926ed-acbf-4dfa-ae54-e2d6b705f1e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:53.699268 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652926ed-acbf-4dfa-ae54-e2d6b705f1e7 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:53.699846 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc67700-360c-4d00-9baf-e1e8f16ea6f3 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:53.712843 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 902/1799] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:53.714144 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652926ed-acbf-4dfa-ae54-e2d6b705f1e7 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:53.716358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-652926ed-acbf-4dfa-ae54-e2d6b705f1e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:53.732771 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652926ed-acbf-4dfa-ae54-e2d6b705f1e7 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:53.733248 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 898/1800] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:13:53 2022] POST /identity/v3/users => generated 292 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:13:53.734339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc67700-360c-4d00-9baf-e1e8f16ea6f3 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:53.737437 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd46ba31-c5ab-47b8-b06a-332a4eac1412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:53.738061 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46ba31-c5ab-47b8-b06a-332a4eac1412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:53.738446 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46ba31-c5ab-47b8-b06a-332a4eac1412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:53.738721 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46ba31-c5ab-47b8-b06a-332a4eac1412 admin admin] PATH_INFO: `/v3/users/d367dcbee8fa49feb6dc0a6a07f85dfd` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:53.739923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd46ba31-c5ab-47b8-b06a-332a4eac1412 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d367dcbee8fa49feb6dc0a6a07f85dfd)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:53.741969 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd46ba31-c5ab-47b8-b06a-332a4eac1412 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:53.745439 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc67700-360c-4d00-9baf-e1e8f16ea6f3 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:53.751708 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8de427-8d5f-4ad3-9b8d-339fee0d7973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:53.752425 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8de427-8d5f-4ad3-9b8d-339fee0d7973 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:53.752637 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8de427-8d5f-4ad3-9b8d-339fee0d7973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:53.752911 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8de427-8d5f-4ad3-9b8d-339fee0d7973 admin admin] PATH_INFO: `/v3/projects/6afb1608b060494b8be8719ea6d77119/users/8c33fd0b6678481f99802b8d513fd0b2/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:53.768687 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8de427-8d5f-4ad3-9b8d-339fee0d7973 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6afb1608b060494b8be8719ea6d77119, user_id=8c33fd0b6678481f99802b8d513fd0b2, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:53.771391 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8de427-8d5f-4ad3-9b8d-339fee0d7973 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:53.784245 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 899/1801] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:13:53 2022] PUT /identity/v3/projects/6afb1608b060494b8be8719ea6d77119/users/8c33fd0b6678481f99802b8d513fd0b2/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:53.785058 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8de427-8d5f-4ad3-9b8d-339fee0d7973 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:53.785539 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:53.795401 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-bd46ba31-c5ab-47b8-b06a-332a4eac1412 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd367dcbee8fa49feb6dc0a6a07f85dfd'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:53.803907 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-bd46ba31-c5ab-47b8-b06a-332a4eac1412 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd367dcbee8fa49feb6dc0a6a07f85dfd'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:53.804599 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-bd46ba31-c5ab-47b8-b06a-332a4eac1412 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd367dcbee8fa49feb6dc0a6a07f85dfd'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:53.805470 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-bd46ba31-c5ab-47b8-b06a-332a4eac1412 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd367dcbee8fa49feb6dc0a6a07f85dfd'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:53.813478 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-bd46ba31-c5ab-47b8-b06a-332a4eac1412 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd367dcbee8fa49feb6dc0a6a07f85dfd'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:53.840703 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 903/1802] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:13:53 2022] DELETE /identity/v3/users/d367dcbee8fa49feb6dc0a6a07f85dfd => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:53.844052 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d1c7d-c0ff-4e15-8098-dd9f84763638 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:53.844052 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d1c7d-c0ff-4e15-8098-dd9f84763638 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:53.844052 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d1c7d-c0ff-4e15-8098-dd9f84763638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:53.849419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8de427-8d5f-4ad3-9b8d-339fee0d7973 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:53.871808 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-1f1d1c7d-c0ff-4e15-8098-dd9f84763638 None None] MFA Rules not processed for user `8c33fd0b6678481f99802b8d513fd0b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:53.873026 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1f1d1c7d-c0ff-4e15-8098-dd9f84763638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:53.908393 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d0d7f-a8c5-4695-92cd-69f31d5b906e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:53.908393 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d0d7f-a8c5-4695-92cd-69f31d5b906e admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:53.908684 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d0d7f-a8c5-4695-92cd-69f31d5b906e admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:53.908785 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d0d7f-a8c5-4695-92cd-69f31d5b906e admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:53.909394 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1d0d7f-a8c5-4695-92cd-69f31d5b906e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:53.911008 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1d0d7f-a8c5-4695-92cd-69f31d5b906e admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:53.927380 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 900/1803] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:53.927380 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7883a-a6c7-4f4e-a129-958c1efd74bb None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:53.927380 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7883a-a6c7-4f4e-a129-958c1efd74bb None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:53.927380 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7883a-a6c7-4f4e-a129-958c1efd74bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:53.943916 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 904/1804] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:53 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:53.944789 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d1c7d-c0ff-4e15-8098-dd9f84763638 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:53.949341 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1f1d1c7d-c0ff-4e15-8098-dd9f84763638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:53.956254 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-8ce7883a-a6c7-4f4e-a129-958c1efd74bb None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:53.956907 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-8ce7883a-a6c7-4f4e-a129-958c1efd74bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:53.965445 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 901/1805] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:53.969865 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce7883a-a6c7-4f4e-a129-958c1efd74bb None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:53.973731 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-8ce7883a-a6c7-4f4e-a129-958c1efd74bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:53.978573 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d1c7d-c0ff-4e15-8098-dd9f84763638 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:53.984516 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db72e909-e858-43ad-b03c-d7994066d23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:53.985526 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72e909-e858-43ad-b03c-d7994066d23b admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:53.985915 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72e909-e858-43ad-b03c-d7994066d23b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:53.986279 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72e909-e858-43ad-b03c-d7994066d23b admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:53.987077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db72e909-e858-43ad-b03c-d7994066d23b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:53.988811 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db72e909-e858-43ad-b03c-d7994066d23b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:53.989674 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce7883a-a6c7-4f4e-a129-958c1efd74bb None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:53.994780 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c59c896-685a-4b71-8274-03833eb162ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:53.995444 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59c896-685a-4b71-8274-03833eb162ae admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:53.995719 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59c896-685a-4b71-8274-03833eb162ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:53.996022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59c896-685a-4b71-8274-03833eb162ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:53.996665 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c59c896-685a-4b71-8274-03833eb162ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:53.999592 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c59c896-685a-4b71-8274-03833eb162ae admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:54.004607 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 905/1806] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:54.005638 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db72e909-e858-43ad-b03c-d7994066d23b admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:54.007492 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-db72e909-e858-43ad-b03c-d7994066d23b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:54.044044 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 902/1807] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:13:53 2022] POST /identity/v3/projects => generated 401 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:54.045516 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c59c896-685a-4b71-8274-03833eb162ae admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:54.088180 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db72e909-e858-43ad-b03c-d7994066d23b admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:54.098658 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:54.131278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c59c896-685a-4b71-8274-03833eb162ae admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:54.137376 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45f209c-ac40-4961-bb6b-61110fbb34ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:54.138301 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f209c-ac40-4961-bb6b-61110fbb34ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:54.138795 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f209c-ac40-4961-bb6b-61110fbb34ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:54.139296 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f209c-ac40-4961-bb6b-61110fbb34ba admin admin] PATH_INFO: `/v3/projects/61f51cf3805e402c983f898de32034f0/users/cb91ed19824f4b179be7457a2edc88ae/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:54.150975 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:54.157953 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d8df8f-9495-48cc-bfc4-a5c798c72d4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:54.159593 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8df8f-9495-48cc-bfc4-a5c798c72d4b service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:54.160369 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8df8f-9495-48cc-bfc4-a5c798c72d4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:54.161428 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8df8f-9495-48cc-bfc4-a5c798c72d4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:54.164188 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45f209c-ac40-4961-bb6b-61110fbb34ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=61f51cf3805e402c983f898de32034f0, user_id=cb91ed19824f4b179be7457a2edc88ae, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:54.168589 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45f209c-ac40-4961-bb6b-61110fbb34ba admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:54.176354 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d8df8f-9495-48cc-bfc4-a5c798c72d4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:54.178100 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d8df8f-9495-48cc-bfc4-a5c798c72d4b service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:54.191424 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 903/1808] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:13:54 2022] PUT /identity/v3/projects/61f51cf3805e402c983f898de32034f0/users/cb91ed19824f4b179be7457a2edc88ae/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:54.193237 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45f209c-ac40-4961-bb6b-61110fbb34ba admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:54.193347 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 906/1809] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:13:54.203349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85973922-d441-4a89-8ba2-b55a07906259 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:54.203349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85973922-d441-4a89-8ba2-b55a07906259 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:54.203678 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85973922-d441-4a89-8ba2-b55a07906259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:54.244561 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-85973922-d441-4a89-8ba2-b55a07906259 None None] MFA Rules not processed for user `cb91ed19824f4b179be7457a2edc88ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:54.246576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-85973922-d441-4a89-8ba2-b55a07906259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:54.272330 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45f209c-ac40-4961-bb6b-61110fbb34ba admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:54.278050 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c2719b-5a94-4ee8-ac0b-44fbe007bd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:54.278884 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2719b-5a94-4ee8-ac0b-44fbe007bd2b admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:54.279279 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2719b-5a94-4ee8-ac0b-44fbe007bd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:54.279635 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2719b-5a94-4ee8-ac0b-44fbe007bd2b admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:54.280606 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c2719b-5a94-4ee8-ac0b-44fbe007bd2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:54.284385 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c2719b-5a94-4ee8-ac0b-44fbe007bd2b admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:54.313439 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 904/1810] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:13:54 2022] POST /identity/v3/users => generated 307 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:54.317764 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c2719b-5a94-4ee8-ac0b-44fbe007bd2b admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:54.331912 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c2719b-5a94-4ee8-ac0b-44fbe007bd2b admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:54.337685 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67c719-5a92-494e-816b-82ce9d96fc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:54.338560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67c719-5a92-494e-816b-82ce9d96fc65 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:54.338935 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67c719-5a92-494e-816b-82ce9d96fc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:54.339306 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67c719-5a92-494e-816b-82ce9d96fc65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:54.343375 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e67c719-5a92-494e-816b-82ce9d96fc65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:54.345866 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e67c719-5a92-494e-816b-82ce9d96fc65 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:54.352899 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 907/1811] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:54.358878 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 905/1812] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:54.363627 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67c719-5a92-494e-816b-82ce9d96fc65 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:54.363777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85973922-d441-4a89-8ba2-b55a07906259 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:54.376475 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85973922-d441-4a89-8ba2-b55a07906259 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:54.377106 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67c719-5a92-494e-816b-82ce9d96fc65 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:54.383055 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41ce0a-e124-4c06-831c-eb0cb82bb44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:54.383629 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c454ee-06cc-4317-b086-03c8e632ef1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:54.383912 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41ce0a-e124-4c06-831c-eb0cb82bb44b admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:54.384263 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41ce0a-e124-4c06-831c-eb0cb82bb44b admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:54.384604 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41ce0a-e124-4c06-831c-eb0cb82bb44b admin admin] PATH_INFO: `/v3/projects/47bea51bd78b4b618c68c32b9c5341dc/users/af42d681a8d646b889b47e65a9dc2726/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:54.384722 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c454ee-06cc-4317-b086-03c8e632ef1c admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:54.385115 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c454ee-06cc-4317-b086-03c8e632ef1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:54.385752 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c454ee-06cc-4317-b086-03c8e632ef1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:54.386478 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c454ee-06cc-4317-b086-03c8e632ef1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:54.389898 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c454ee-06cc-4317-b086-03c8e632ef1c admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:54.403024 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec41ce0a-e124-4c06-831c-eb0cb82bb44b admin admin] RBAC: Authorizing `identity:create_grant(project_id=47bea51bd78b4b618c68c32b9c5341dc, user_id=af42d681a8d646b889b47e65a9dc2726, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:54.406337 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec41ce0a-e124-4c06-831c-eb0cb82bb44b admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:54.430331 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 906/1813] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:13:54 2022] PUT /identity/v3/projects/47bea51bd78b4b618c68c32b9c5341dc/users/af42d681a8d646b889b47e65a9dc2726/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:54.431944 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41ce0a-e124-4c06-831c-eb0cb82bb44b admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:54.435835 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 908/1814] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:13:54 2022] POST /identity/v3/projects => generated 385 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:54.439814 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c454ee-06cc-4317-b086-03c8e632ef1c admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:54.543810 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c454ee-06cc-4317-b086-03c8e632ef1c admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:54.549660 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a20cec9-0c53-4a5e-bf88-e3a0877b5a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:54.550384 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20cec9-0c53-4a5e-bf88-e3a0877b5a42 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:54.550780 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20cec9-0c53-4a5e-bf88-e3a0877b5a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:54.551179 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20cec9-0c53-4a5e-bf88-e3a0877b5a42 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:54.551912 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a20cec9-0c53-4a5e-bf88-e3a0877b5a42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:54.554236 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a20cec9-0c53-4a5e-bf88-e3a0877b5a42 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:54.561835 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41ce0a-e124-4c06-831c-eb0cb82bb44b admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:54.571901 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834e0098-065f-417e-a39c-ed965087156e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:54.573934 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e0098-065f-417e-a39c-ed965087156e admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:54.576204 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e0098-065f-417e-a39c-ed965087156e admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:54.576667 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e0098-065f-417e-a39c-ed965087156e admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:54.579796 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834e0098-065f-417e-a39c-ed965087156e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:54.581532 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834e0098-065f-417e-a39c-ed965087156e admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:54.607958 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 909/1815] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:13:54 2022] POST /identity/v3/users => generated 299 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:54.611749 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 907/1816] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:54 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:54.614301 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834e0098-065f-417e-a39c-ed965087156e admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:54.614449 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a20cec9-0c53-4a5e-bf88-e3a0877b5a42 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:54.632123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a20cec9-0c53-4a5e-bf88-e3a0877b5a42 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:54.637903 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834e0098-065f-417e-a39c-ed965087156e admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:54.638502 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b9941-af8d-41ce-b64f-51bbc1bb8929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:54.639360 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b9941-af8d-41ce-b64f-51bbc1bb8929 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:54.639754 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b9941-af8d-41ce-b64f-51bbc1bb8929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:54.640115 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b9941-af8d-41ce-b64f-51bbc1bb8929 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:54.641442 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b9941-af8d-41ce-b64f-51bbc1bb8929 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:54.643427 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b9941-af8d-41ce-b64f-51bbc1bb8929 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:54.653987 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358060a3-b06b-4fa4-9f5c-8dde5019abb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:54.654681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358060a3-b06b-4fa4-9f5c-8dde5019abb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:54.654995 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358060a3-b06b-4fa4-9f5c-8dde5019abb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:54.655275 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358060a3-b06b-4fa4-9f5c-8dde5019abb9 admin admin] PATH_INFO: `/v3/projects/47bea51bd78b4b618c68c32b9c5341dc/users/af42d681a8d646b889b47e65a9dc2726/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:54.660751 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358060a3-b06b-4fa4-9f5c-8dde5019abb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47bea51bd78b4b618c68c32b9c5341dc, user_id=af42d681a8d646b889b47e65a9dc2726, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:54.660942 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 910/1817] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:54 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:54.663523 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358060a3-b06b-4fa4-9f5c-8dde5019abb9 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:54.664663 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b9941-af8d-41ce-b64f-51bbc1bb8929 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:54.675619 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b9941-af8d-41ce-b64f-51bbc1bb8929 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:54.678507 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:54.680816 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c558a-eae6-4756-a82e-70cf8527c8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:54.682078 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c558a-eae6-4756-a82e-70cf8527c8c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:54.682078 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c558a-eae6-4756-a82e-70cf8527c8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:54.682391 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c558a-eae6-4756-a82e-70cf8527c8c8 admin admin] PATH_INFO: `/v3/projects/f9cf3d317d554827b02ab4d69481c893/users/d9337117e8ed4e41a2d6514d9913750a/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:54.694357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 908/1818] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:13:54 2022] PUT /identity/v3/projects/47bea51bd78b4b618c68c32b9c5341dc/users/af42d681a8d646b889b47e65a9dc2726/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:54.706081 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55c558a-eae6-4756-a82e-70cf8527c8c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9cf3d317d554827b02ab4d69481c893, user_id=d9337117e8ed4e41a2d6514d9913750a, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:54.707497 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681159a8-5fef-4ee3-a7c0-0d7778953a35 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:54.707785 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681159a8-5fef-4ee3-a7c0-0d7778953a35 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:54.708057 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681159a8-5fef-4ee3-a7c0-0d7778953a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:54.708815 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55c558a-eae6-4756-a82e-70cf8527c8c8 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:54.728712 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 911/1819] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:13:54 2022] PUT /identity/v3/projects/f9cf3d317d554827b02ab4d69481c893/users/d9337117e8ed4e41a2d6514d9913750a/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:54.729962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c558a-eae6-4756-a82e-70cf8527c8c8 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:54.743867 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-681159a8-5fef-4ee3-a7c0-0d7778953a35 None None] MFA Rules not processed for user `af42d681a8d646b889b47e65a9dc2726`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:54.745255 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-681159a8-5fef-4ee3-a7c0-0d7778953a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:54.749565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:54.810759 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c558a-eae6-4756-a82e-70cf8527c8c8 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:54.817105 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c20d8-a512-48c3-b5eb-01635d3f46d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:54.817883 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c20d8-a512-48c3-b5eb-01635d3f46d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:54.820807 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c20d8-a512-48c3-b5eb-01635d3f46d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:54.821254 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c20d8-a512-48c3-b5eb-01635d3f46d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:54.822126 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857c20d8-a512-48c3-b5eb-01635d3f46d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:54.823894 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857c20d8-a512-48c3-b5eb-01635d3f46d4 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:54.840561 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 912/1820] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:54 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:13:54.842005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c20d8-a512-48c3-b5eb-01635d3f46d4 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:54.854021 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c20d8-a512-48c3-b5eb-01635d3f46d4 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:54.858997 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 909/1821] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:54 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:54.859596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98884ac9-1d21-497c-897d-71769b809e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:54.860012 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681159a8-5fef-4ee3-a7c0-0d7778953a35 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:54.860673 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98884ac9-1d21-497c-897d-71769b809e22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:54.861484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98884ac9-1d21-497c-897d-71769b809e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:54.861818 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98884ac9-1d21-497c-897d-71769b809e22 admin admin] PATH_INFO: `/v3/projects/f9cf3d317d554827b02ab4d69481c893/users/d9337117e8ed4e41a2d6514d9913750a/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:54.862385 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-681159a8-5fef-4ee3-a7c0-0d7778953a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:54.864364 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98884ac9-1d21-497c-897d-71769b809e22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9cf3d317d554827b02ab4d69481c893, user_id=d9337117e8ed4e41a2d6514d9913750a, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:54.867360 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98884ac9-1d21-497c-897d-71769b809e22 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:54.886935 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 913/1822] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:13:54 2022] PUT /identity/v3/projects/f9cf3d317d554827b02ab4d69481c893/users/d9337117e8ed4e41a2d6514d9913750a/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:54.894520 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0de59-ed98-47aa-9c47-415e824cf74c None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:54.897666 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0de59-ed98-47aa-9c47-415e824cf74c None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:54.898215 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0de59-ed98-47aa-9c47-415e824cf74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:54.939786 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-eab0de59-ed98-47aa-9c47-415e824cf74c None None] MFA Rules not processed for user `d9337117e8ed4e41a2d6514d9913750a`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:54.941892 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-eab0de59-ed98-47aa-9c47-415e824cf74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:54.997095 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681159a8-5fef-4ee3-a7c0-0d7778953a35 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:55.048532 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 914/1823] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:54 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:55.059848 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0de59-ed98-47aa-9c47-415e824cf74c None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:55.061759 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-eab0de59-ed98-47aa-9c47-415e824cf74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:55.084442 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5dbf2f-b61c-4249-a591-29046ba6e1df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:55.085214 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5dbf2f-b61c-4249-a591-29046ba6e1df service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:55.085599 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5dbf2f-b61c-4249-a591-29046ba6e1df service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:55.085974 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5dbf2f-b61c-4249-a591-29046ba6e1df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:55.150768 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:55.160404 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0de59-ed98-47aa-9c47-415e824cf74c None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:55.164459 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db4224-e832-4fcd-84a8-de226f688414 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:55.165080 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db4224-e832-4fcd-84a8-de226f688414 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:55.165494 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db4224-e832-4fcd-84a8-de226f688414 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:55.165775 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db4224-e832-4fcd-84a8-de226f688414 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:55.168028 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5dbf2f-b61c-4249-a591-29046ba6e1df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:55.169684 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5dbf2f-b61c-4249-a591-29046ba6e1df service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:55.175871 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12db4224-e832-4fcd-84a8-de226f688414 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:55.178337 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12db4224-e832-4fcd-84a8-de226f688414 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:55.183495 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 910/1824] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:54 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:13:55.190723 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 915/1825] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:55 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:13:55.287918 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5dbf2f-b61c-4249-a591-29046ba6e1df service cinder] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:55.296598 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5dbf2f-b61c-4249-a591-29046ba6e1df service cinder] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:55.303074 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1842a-048c-4adc-bfd5-321854e75a28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:55.303705 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1842a-048c-4adc-bfd5-321854e75a28 service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:55.304047 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1842a-048c-4adc-bfd5-321854e75a28 service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:55.304328 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1842a-048c-4adc-bfd5-321854e75a28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:55.379610 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1842a-048c-4adc-bfd5-321854e75a28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:55.381139 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1842a-048c-4adc-bfd5-321854e75a28 service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:55.411047 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 911/1826] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:55 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:13:55.817740 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db4224-e832-4fcd-84a8-de226f688414 service cinder] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:55.819565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-12db4224-e832-4fcd-84a8-de226f688414 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:55.864388 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:55.966971 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db4224-e832-4fcd-84a8-de226f688414 service cinder] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:55.982468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90244eae-e9a4-40b5-b5ba-f45142c55ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:55.983100 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90244eae-e9a4-40b5-b5ba-f45142c55ea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:55.983978 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90244eae-e9a4-40b5-b5ba-f45142c55ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:55.985201 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90244eae-e9a4-40b5-b5ba-f45142c55ea3 admin admin] PATH_INFO: `/v3/projects/30b7cb46a2234ea9b928a3fc48abddfa` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:55.986797 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90244eae-e9a4-40b5-b5ba-f45142c55ea3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30b7cb46a2234ea9b928a3fc48abddfa)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:55.990259 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90244eae-e9a4-40b5-b5ba-f45142c55ea3 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:56.117409 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-90244eae-e9a4-40b5-b5ba-f45142c55ea3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30b7cb46a2234ea9b928a3fc48abddfa', 'request_id': 'req-90244eae-e9a4-40b5-b5ba-f45142c55ea3'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:56.150374 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-90244eae-e9a4-40b5-b5ba-f45142c55ea3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30b7cb46a2234ea9b928a3fc48abddfa', 'request_id': 'req-90244eae-e9a4-40b5-b5ba-f45142c55ea3'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:56.159238 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-90244eae-e9a4-40b5-b5ba-f45142c55ea3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30b7cb46a2234ea9b928a3fc48abddfa', 'request_id': 'req-90244eae-e9a4-40b5-b5ba-f45142c55ea3'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:56.178096 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-90244eae-e9a4-40b5-b5ba-f45142c55ea3 admin admin] The token cache is being invalidate because project 30b7cb46a2234ea9b928a3fc48abddfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:13:56.178394 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-90244eae-e9a4-40b5-b5ba-f45142c55ea3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:56.179177 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-90244eae-e9a4-40b5-b5ba-f45142c55ea3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:56.181584 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 916/1827] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:13:55 2022] DELETE /identity/v3/projects/30b7cb46a2234ea9b928a3fc48abddfa => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:13:56.185862 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1842a-048c-4adc-bfd5-321854e75a28 service cinder] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:56.204366 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1842a-048c-4adc-bfd5-321854e75a28 service cinder] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:56.240637 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90244eae-e9a4-40b5-b5ba-f45142c55ea3 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:56.242127 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-90244eae-e9a4-40b5-b5ba-f45142c55ea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:56.284609 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da9dec-5f44-44bf-adc2-a71fd4cd6b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:56.285645 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da9dec-5f44-44bf-adc2-a71fd4cd6b20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:56.286173 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da9dec-5f44-44bf-adc2-a71fd4cd6b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:56.286751 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da9dec-5f44-44bf-adc2-a71fd4cd6b20 admin admin] PATH_INFO: `/v3/users/8d8e02415e854b4a92bd457dba02e1f9` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:56.288170 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da9dec-5f44-44bf-adc2-a71fd4cd6b20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d8e02415e854b4a92bd457dba02e1f9)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:56.290363 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da9dec-5f44-44bf-adc2-a71fd4cd6b20 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:56.345937 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90244eae-e9a4-40b5-b5ba-f45142c55ea3 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:56.354566 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f42310-6932-4f6a-bebc-fa4ce5f0abcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:56.355246 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f42310-6932-4f6a-bebc-fa4ce5f0abcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:56.355525 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f42310-6932-4f6a-bebc-fa4ce5f0abcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:56.355800 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f42310-6932-4f6a-bebc-fa4ce5f0abcc admin admin] PATH_INFO: `/v3/users/cb91ed19824f4b179be7457a2edc88ae` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:56.357767 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f42310-6932-4f6a-bebc-fa4ce5f0abcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb91ed19824f4b179be7457a2edc88ae)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:56.359241 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f42310-6932-4f6a-bebc-fa4ce5f0abcc admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:56.376996 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-57da9dec-5f44-44bf-adc2-a71fd4cd6b20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d8e02415e854b4a92bd457dba02e1f9'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:56.385980 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-57da9dec-5f44-44bf-adc2-a71fd4cd6b20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d8e02415e854b4a92bd457dba02e1f9'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:56.386628 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-57da9dec-5f44-44bf-adc2-a71fd4cd6b20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d8e02415e854b4a92bd457dba02e1f9'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:56.390011 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-57da9dec-5f44-44bf-adc2-a71fd4cd6b20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d8e02415e854b4a92bd457dba02e1f9'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:56.415982 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-57da9dec-5f44-44bf-adc2-a71fd4cd6b20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d8e02415e854b4a92bd457dba02e1f9'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:56.440803 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e1f42310-6932-4f6a-bebc-fa4ce5f0abcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb91ed19824f4b179be7457a2edc88ae'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:56.457101 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e1f42310-6932-4f6a-bebc-fa4ce5f0abcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb91ed19824f4b179be7457a2edc88ae'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:56.459024 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e1f42310-6932-4f6a-bebc-fa4ce5f0abcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb91ed19824f4b179be7457a2edc88ae'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:56.460065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e1f42310-6932-4f6a-bebc-fa4ce5f0abcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb91ed19824f4b179be7457a2edc88ae'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:56.464733 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 912/1828] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:13:56 2022] DELETE /identity/v3/users/8d8e02415e854b4a92bd457dba02e1f9 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:56.468627 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:56.477572 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e1f42310-6932-4f6a-bebc-fa4ce5f0abcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb91ed19824f4b179be7457a2edc88ae'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:56.500109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 917/1829] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:13:56 2022] DELETE /identity/v3/users/cb91ed19824f4b179be7457a2edc88ae => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:13:56.504951 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707f91f-d6a3-4e5e-9289-6c9fa845d0fb None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:56.505234 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707f91f-d6a3-4e5e-9289-6c9fa845d0fb None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:56.505589 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707f91f-d6a3-4e5e-9289-6c9fa845d0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:56.514668 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:56.528394 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-c707f91f-d6a3-4e5e-9289-6c9fa845d0fb None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:56.529758 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-c707f91f-d6a3-4e5e-9289-6c9fa845d0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:56.607309 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 913/1830] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:56.617381 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f42310-6932-4f6a-bebc-fa4ce5f0abcc admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:56.619386 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e1f42310-6932-4f6a-bebc-fa4ce5f0abcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:56.702147 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f42310-6932-4f6a-bebc-fa4ce5f0abcc admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:56.706828 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b2101-ae0f-4b3e-88ea-5265e5ec3f4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:56.707506 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b2101-ae0f-4b3e-88ea-5265e5ec3f4e service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:56.707872 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b2101-ae0f-4b3e-88ea-5265e5ec3f4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:56.708162 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b2101-ae0f-4b3e-88ea-5265e5ec3f4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:56.718875 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b2101-ae0f-4b3e-88ea-5265e5ec3f4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:56.722738 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b2101-ae0f-4b3e-88ea-5265e5ec3f4e service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:56.741415 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 918/1831] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:13:57.069937 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c707f91f-d6a3-4e5e-9289-6c9fa845d0fb None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:57.071698 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-c707f91f-d6a3-4e5e-9289-6c9fa845d0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:57.085744 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c707f91f-d6a3-4e5e-9289-6c9fa845d0fb None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:57.090694 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b6e1c-b9e9-4d6e-89ac-1521748d70da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:57.091922 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b6e1c-b9e9-4d6e-89ac-1521748d70da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:57.091922 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b6e1c-b9e9-4d6e-89ac-1521748d70da admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:57.092147 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b6e1c-b9e9-4d6e-89ac-1521748d70da admin admin] PATH_INFO: `/v3/projects/dde71a33c3e54e14812725ef913bf2e1` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:57.093371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561b6e1c-b9e9-4d6e-89ac-1521748d70da admin admin] RBAC: Authorizing `identity:delete_project(project_id=dde71a33c3e54e14812725ef913bf2e1)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:57.095911 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561b6e1c-b9e9-4d6e-89ac-1521748d70da admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:57.169411 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-561b6e1c-b9e9-4d6e-89ac-1521748d70da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dde71a33c3e54e14812725ef913bf2e1', 'request_id': 'req-561b6e1c-b9e9-4d6e-89ac-1521748d70da'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:57.176448 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-561b6e1c-b9e9-4d6e-89ac-1521748d70da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dde71a33c3e54e14812725ef913bf2e1', 'request_id': 'req-561b6e1c-b9e9-4d6e-89ac-1521748d70da'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:57.184586 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-561b6e1c-b9e9-4d6e-89ac-1521748d70da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dde71a33c3e54e14812725ef913bf2e1', 'request_id': 'req-561b6e1c-b9e9-4d6e-89ac-1521748d70da'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:57.192984 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b2101-ae0f-4b3e-88ea-5265e5ec3f4e service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:57.194464 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-a84b2101-ae0f-4b3e-88ea-5265e5ec3f4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:57.204217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-561b6e1c-b9e9-4d6e-89ac-1521748d70da admin admin] The token cache is being invalidate because project dde71a33c3e54e14812725ef913bf2e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:13:57.205321 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-561b6e1c-b9e9-4d6e-89ac-1521748d70da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:57.206806 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-561b6e1c-b9e9-4d6e-89ac-1521748d70da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:57.212866 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 914/1832] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:13:57 2022] DELETE /identity/v3/projects/dde71a33c3e54e14812725ef913bf2e1 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:57.214917 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85acc0-1692-47f5-b612-3e2e3dbd8c07 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:57.215442 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85acc0-1692-47f5-b612-3e2e3dbd8c07 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:57.215818 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85acc0-1692-47f5-b612-3e2e3dbd8c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:57.222536 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b2101-ae0f-4b3e-88ea-5265e5ec3f4e service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:57.247065 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-ad85acc0-1692-47f5-b612-3e2e3dbd8c07 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:57.247065 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-ad85acc0-1692-47f5-b612-3e2e3dbd8c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:57.319029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 915/1833] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:57.330511 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ff17e-6113-44bf-b866-e105aa1ba479 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:57.330695 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ff17e-6113-44bf-b866-e105aa1ba479 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:57.331129 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ff17e-6113-44bf-b866-e105aa1ba479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:57.377744 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aabee81-fcc5-4a90-bbaa-14d361e8061d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:57.378806 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aabee81-fcc5-4a90-bbaa-14d361e8061d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:57.379266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aabee81-fcc5-4a90-bbaa-14d361e8061d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:57.379645 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aabee81-fcc5-4a90-bbaa-14d361e8061d admin admin] PATH_INFO: `/v3/projects/61f51cf3805e402c983f898de32034f0` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:57.381215 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aabee81-fcc5-4a90-bbaa-14d361e8061d admin admin] RBAC: Authorizing `identity:delete_project(project_id=61f51cf3805e402c983f898de32034f0)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:57.383641 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aabee81-fcc5-4a90-bbaa-14d361e8061d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:57.385119 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-836ff17e-6113-44bf-b866-e105aa1ba479 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:57.394555 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-836ff17e-6113-44bf-b866-e105aa1ba479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:57.435151 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 916/1834] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:57.436688 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ff17e-6113-44bf-b866-e105aa1ba479 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:57.499648 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ff17e-6113-44bf-b866-e105aa1ba479 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:57.538338 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5468e-4d04-4a42-aba8-bb8fc494a68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:57.539350 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5468e-4d04-4a42-aba8-bb8fc494a68c admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:57.541279 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5468e-4d04-4a42-aba8-bb8fc494a68c admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:57.543438 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5468e-4d04-4a42-aba8-bb8fc494a68c admin admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:57.544392 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e5468e-4d04-4a42-aba8-bb8fc494a68c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:13:57.547675 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e5468e-4d04-4a42-aba8-bb8fc494a68c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:57.551617 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e5468e-4d04-4a42-aba8-bb8fc494a68c admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:57.572445 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 917/1835] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:13:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:13:57.577102 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-5aabee81-fcc5-4a90-bbaa-14d361e8061d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61f51cf3805e402c983f898de32034f0', 'request_id': 'req-5aabee81-fcc5-4a90-bbaa-14d361e8061d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:57.577923 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5468e-4d04-4a42-aba8-bb8fc494a68c admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:57.591784 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5468e-4d04-4a42-aba8-bb8fc494a68c admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:57.592476 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-5aabee81-fcc5-4a90-bbaa-14d361e8061d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61f51cf3805e402c983f898de32034f0', 'request_id': 'req-5aabee81-fcc5-4a90-bbaa-14d361e8061d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:57.601353 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:57.604539 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-5aabee81-fcc5-4a90-bbaa-14d361e8061d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61f51cf3805e402c983f898de32034f0', 'request_id': 'req-5aabee81-fcc5-4a90-bbaa-14d361e8061d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:57.618517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-5aabee81-fcc5-4a90-bbaa-14d361e8061d admin admin] The token cache is being invalidate because project 61f51cf3805e402c983f898de32034f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:13:57.619109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-5aabee81-fcc5-4a90-bbaa-14d361e8061d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:57.619930 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-5aabee81-fcc5-4a90-bbaa-14d361e8061d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:57.621718 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 919/1836] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:13:57 2022] DELETE /identity/v3/projects/61f51cf3805e402c983f898de32034f0 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:13:57.626035 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aabee81-fcc5-4a90-bbaa-14d361e8061d admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:57.627417 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-5aabee81-fcc5-4a90-bbaa-14d361e8061d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:57.640257 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:57.679396 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d4acc4-fc37-4eff-823e-536e570826f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:57.680221 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d4acc4-fc37-4eff-823e-536e570826f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:57.680600 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d4acc4-fc37-4eff-823e-536e570826f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:57.680967 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d4acc4-fc37-4eff-823e-536e570826f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:57.681842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d4acc4-fc37-4eff-823e-536e570826f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:57.684115 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d4acc4-fc37-4eff-823e-536e570826f1 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:57.725914 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aabee81-fcc5-4a90-bbaa-14d361e8061d admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:57.725914 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc9d1e2-e7ca-4b74-80de-dd4a2012e450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:57.725914 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9d1e2-e7ca-4b74-80de-dd4a2012e450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:57.725914 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9d1e2-e7ca-4b74-80de-dd4a2012e450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:57.726444 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9d1e2-e7ca-4b74-80de-dd4a2012e450 admin admin] PATH_INFO: `/v3/users/d9337117e8ed4e41a2d6514d9913750a` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:57.726636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc9d1e2-e7ca-4b74-80de-dd4a2012e450 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9337117e8ed4e41a2d6514d9913750a)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:57.730259 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc9d1e2-e7ca-4b74-80de-dd4a2012e450 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:57.744903 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 918/1837] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:13:57 2022] POST /identity/v3/projects => generated 381 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:57.749993 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d4acc4-fc37-4eff-823e-536e570826f1 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:57.752694 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-96d4acc4-fc37-4eff-823e-536e570826f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:57.815882 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-5fc9d1e2-e7ca-4b74-80de-dd4a2012e450 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9337117e8ed4e41a2d6514d9913750a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:57.837306 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-5fc9d1e2-e7ca-4b74-80de-dd4a2012e450 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9337117e8ed4e41a2d6514d9913750a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:57.838682 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-5fc9d1e2-e7ca-4b74-80de-dd4a2012e450 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9337117e8ed4e41a2d6514d9913750a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:57.840124 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-5fc9d1e2-e7ca-4b74-80de-dd4a2012e450 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9337117e8ed4e41a2d6514d9913750a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:57.856951 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-5fc9d1e2-e7ca-4b74-80de-dd4a2012e450 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9337117e8ed4e41a2d6514d9913750a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:57.858815 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d4acc4-fc37-4eff-823e-536e570826f1 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:57.884297 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 920/1838] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:13:57 2022] DELETE /identity/v3/users/d9337117e8ed4e41a2d6514d9913750a => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:13:57.886966 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f888d2c-fe32-4910-aff7-9f7d63d67300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:57.887674 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f888d2c-fe32-4910-aff7-9f7d63d67300 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:57.888039 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f888d2c-fe32-4910-aff7-9f7d63d67300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:57.888369 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f888d2c-fe32-4910-aff7-9f7d63d67300 admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:57.889076 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f888d2c-fe32-4910-aff7-9f7d63d67300 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:57.894453 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f888d2c-fe32-4910-aff7-9f7d63d67300 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:57.936358 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 919/1839] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:13:57 2022] POST /identity/v3/users => generated 291 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:57.940974 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc9d1e2-e7ca-4b74-80de-dd4a2012e450 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:57.942987 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-5fc9d1e2-e7ca-4b74-80de-dd4a2012e450 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:58.039381 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aceda8-2560-46d5-be91-5795974ba3ee None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:58.040092 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aceda8-2560-46d5-be91-5795974ba3ee None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:58.040387 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aceda8-2560-46d5-be91-5795974ba3ee None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:58.042393 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 920/1840] 10.222.0.42 () {54 vars in 921 bytes} [Mon Sep 12 14:13:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:13:58.058271 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc9d1e2-e7ca-4b74-80de-dd4a2012e450 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:58.063855 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53287031-dcc9-4c75-93f1-7a9e9d73de15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:58.064826 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53287031-dcc9-4c75-93f1-7a9e9d73de15 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:58.065161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53287031-dcc9-4c75-93f1-7a9e9d73de15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:58.065553 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53287031-dcc9-4c75-93f1-7a9e9d73de15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:58.066684 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53287031-dcc9-4c75-93f1-7a9e9d73de15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:58.069717 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53287031-dcc9-4c75-93f1-7a9e9d73de15 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:58.071917 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d580707-74d9-4bba-9bd9-c9ee3deb76c6 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:58.072206 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d580707-74d9-4bba-9bd9-c9ee3deb76c6 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:58.072482 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d580707-74d9-4bba-9bd9-c9ee3deb76c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:58.082484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 921/1841] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:58.086095 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53287031-dcc9-4c75-93f1-7a9e9d73de15 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:58.101550 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53287031-dcc9-4c75-93f1-7a9e9d73de15 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:58.116503 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2841d3fa-1e27-46bf-bfa2-3dc20935419b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:58.117262 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841d3fa-1e27-46bf-bfa2-3dc20935419b admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:58.117902 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841d3fa-1e27-46bf-bfa2-3dc20935419b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:58.118239 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841d3fa-1e27-46bf-bfa2-3dc20935419b admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:58.119610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2841d3fa-1e27-46bf-bfa2-3dc20935419b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:58.119916 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-4d580707-74d9-4bba-9bd9-c9ee3deb76c6 None None] MFA Rules not processed for user `854ab2dcec5a48b18f409b33cb61b5da`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:58.120903 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2841d3fa-1e27-46bf-bfa2-3dc20935419b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:58.121161 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-4d580707-74d9-4bba-9bd9-c9ee3deb76c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:58.139793 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 922/1842] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:58.144323 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2841d3fa-1e27-46bf-bfa2-3dc20935419b admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:58.166886 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2841d3fa-1e27-46bf-bfa2-3dc20935419b admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:58.182206 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddaa8e5-c5a8-4381-9fa8-45b670a84257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:58.183110 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddaa8e5-c5a8-4381-9fa8-45b670a84257 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:58.183454 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddaa8e5-c5a8-4381-9fa8-45b670a84257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:58.184658 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddaa8e5-c5a8-4381-9fa8-45b670a84257 admin admin] PATH_INFO: `/v3/projects/bfc5015a437b403c85d9de1ba184b88d/users/8f9b4454165e4be69d3ad6481ae26a17/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:58.208327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:58.225910 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddaa8e5-c5a8-4381-9fa8-45b670a84257 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfc5015a437b403c85d9de1ba184b88d, user_id=8f9b4454165e4be69d3ad6481ae26a17, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:58.230073 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddaa8e5-c5a8-4381-9fa8-45b670a84257 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:58.257007 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 923/1843] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:13:58 2022] PUT /identity/v3/projects/bfc5015a437b403c85d9de1ba184b88d/users/8f9b4454165e4be69d3ad6481ae26a17/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:58.276087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e97e36-8ac9-4502-ba42-4a920aedb647 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:58.276914 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e97e36-8ac9-4502-ba42-4a920aedb647 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:58.277654 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e97e36-8ac9-4502-ba42-4a920aedb647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:58.278685 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 921/1844] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Sep 12 14:13:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:13:58.293916 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d580707-74d9-4bba-9bd9-c9ee3deb76c6 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:58.297928 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-4d580707-74d9-4bba-9bd9-c9ee3deb76c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:58.330067 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-89e97e36-8ac9-4502-ba42-4a920aedb647 None None] MFA Rules not processed for user `8f9b4454165e4be69d3ad6481ae26a17`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:58.331711 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-89e97e36-8ac9-4502-ba42-4a920aedb647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:58.332721 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d580707-74d9-4bba-9bd9-c9ee3deb76c6 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:58.352472 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:58.434010 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e3443-be44-4736-afba-6a343cae23d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:58.435297 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e3443-be44-4736-afba-6a343cae23d5 service nova] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:58.436046 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e3443-be44-4736-afba-6a343cae23d5 service nova] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:58.437073 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e3443-be44-4736-afba-6a343cae23d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:58.460242 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e3443-be44-4736-afba-6a343cae23d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:58.462982 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e3443-be44-4736-afba-6a343cae23d5 service nova] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:58.481025 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 922/1845] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:13:58.482300 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e3443-be44-4736-afba-6a343cae23d5 service nova] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:58.483949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-153e3443-be44-4736-afba-6a343cae23d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:58.506795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 924/1846] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:58.507725 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e97e36-8ac9-4502-ba42-4a920aedb647 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:58.648840 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e3443-be44-4736-afba-6a343cae23d5 service nova] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:58.665771 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e97e36-8ac9-4502-ba42-4a920aedb647 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:58.671328 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a1869-baf2-4e32-8e41-8e636c46fcf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:58.672120 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e4522-7543-4570-93a5-3fc0676e90a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:58.672587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a1869-baf2-4e32-8e41-8e636c46fcf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:58.672812 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e4522-7543-4570-93a5-3fc0676e90a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:58.672910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a1869-baf2-4e32-8e41-8e636c46fcf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:58.673063 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e4522-7543-4570-93a5-3fc0676e90a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:58.673167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a1869-baf2-4e32-8e41-8e636c46fcf3 admin admin] PATH_INFO: `/v3/projects/f9cf3d317d554827b02ab4d69481c893` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:58.673336 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e4522-7543-4570-93a5-3fc0676e90a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:58.674067 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e4522-7543-4570-93a5-3fc0676e90a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:58.674381 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a1869-baf2-4e32-8e41-8e636c46fcf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9cf3d317d554827b02ab4d69481c893)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:58.676258 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e4522-7543-4570-93a5-3fc0676e90a1 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:58.676467 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a1869-baf2-4e32-8e41-8e636c46fcf3 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:58.728802 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 925/1847] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:13:58 2022] POST /identity/v3/projects => generated 379 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:58.730377 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e4522-7543-4570-93a5-3fc0676e90a1 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:58.784038 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-b67a1869-baf2-4e32-8e41-8e636c46fcf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9cf3d317d554827b02ab4d69481c893', 'request_id': 'req-b67a1869-baf2-4e32-8e41-8e636c46fcf3'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:58.791952 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-b67a1869-baf2-4e32-8e41-8e636c46fcf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9cf3d317d554827b02ab4d69481c893', 'request_id': 'req-b67a1869-baf2-4e32-8e41-8e636c46fcf3'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:58.824615 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-b67a1869-baf2-4e32-8e41-8e636c46fcf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9cf3d317d554827b02ab4d69481c893', 'request_id': 'req-b67a1869-baf2-4e32-8e41-8e636c46fcf3'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:58.846102 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-b67a1869-baf2-4e32-8e41-8e636c46fcf3 admin admin] The token cache is being invalidate because project f9cf3d317d554827b02ab4d69481c893 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:13:58.846415 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-b67a1869-baf2-4e32-8e41-8e636c46fcf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:58.849209 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-b67a1869-baf2-4e32-8e41-8e636c46fcf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:13:58.864569 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 923/1848] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:13:58 2022] DELETE /identity/v3/projects/f9cf3d317d554827b02ab4d69481c893 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:58.864774 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee2820-ad68-4f9e-b40c-4e1e3c3f9c3c None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:58.864967 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee2820-ad68-4f9e-b40c-4e1e3c3f9c3c None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:58.865245 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee2820-ad68-4f9e-b40c-4e1e3c3f9c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:58.865795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e4522-7543-4570-93a5-3fc0676e90a1 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:58.872806 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:58.911241 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-16ee2820-ad68-4f9e-b40c-4e1e3c3f9c3c None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:58.912477 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-16ee2820-ad68-4f9e-b40c-4e1e3c3f9c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:58.969557 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87572db1-cdf0-40ad-afbe-f37d038409e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:58.969557 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87572db1-cdf0-40ad-afbe-f37d038409e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:58.969557 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87572db1-cdf0-40ad-afbe-f37d038409e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:58.969557 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87572db1-cdf0-40ad-afbe-f37d038409e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:58.969966 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87572db1-cdf0-40ad-afbe-f37d038409e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:58.969966 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87572db1-cdf0-40ad-afbe-f37d038409e8 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:58.977122 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:59.004785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 926/1849] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:13:58 2022] POST /identity/v3/users => generated 296 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:59.007077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87572db1-cdf0-40ad-afbe-f37d038409e8 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:59.010451 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-87572db1-cdf0-40ad-afbe-f37d038409e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:59.045109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87572db1-cdf0-40ad-afbe-f37d038409e8 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:59.057620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea06e621-932d-4ead-be07-b1ce4a97df01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:59.058754 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea06e621-932d-4ead-be07-b1ce4a97df01 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:59.058898 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea06e621-932d-4ead-be07-b1ce4a97df01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:59.059274 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea06e621-932d-4ead-be07-b1ce4a97df01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:59.064554 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea06e621-932d-4ead-be07-b1ce4a97df01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:59.067082 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea06e621-932d-4ead-be07-b1ce4a97df01 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:59.081980 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 927/1850] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:59.086907 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 924/1851] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:59.086907 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ee2820-ad68-4f9e-b40c-4e1e3c3f9c3c None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:59.096885 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be346a-e350-414f-b73a-526852d5dccd None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:59.097254 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be346a-e350-414f-b73a-526852d5dccd None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:59.097698 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be346a-e350-414f-b73a-526852d5dccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:59.107690 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ee2820-ad68-4f9e-b40c-4e1e3c3f9c3c None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:59.123150 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6247a4-137a-460e-a696-511abbe0dc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:59.123150 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6247a4-137a-460e-a696-511abbe0dc84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:59.123150 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6247a4-137a-460e-a696-511abbe0dc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:59.123465 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6247a4-137a-460e-a696-511abbe0dc84 admin admin] PATH_INFO: `/v3/projects/080bbbce654c4d24bf3a0bfe37ab5d24/users/800ee919d69b4d1c84618e5504ac9826/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:59.125037 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-39be346a-e350-414f-b73a-526852d5dccd None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:59.129681 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-39be346a-e350-414f-b73a-526852d5dccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:59.144740 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 928/1852] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:59.145952 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be346a-e350-414f-b73a-526852d5dccd None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:59.156082 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be346a-e350-414f-b73a-526852d5dccd None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:59.156586 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6247a4-137a-460e-a696-511abbe0dc84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=080bbbce654c4d24bf3a0bfe37ab5d24, user_id=800ee919d69b4d1c84618e5504ac9826, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:59.160922 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6247a4-137a-460e-a696-511abbe0dc84 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:59.162075 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc45b6-af09-44d7-8682-b81ed121380d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:59.163025 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc45b6-af09-44d7-8682-b81ed121380d admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:59.163720 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc45b6-af09-44d7-8682-b81ed121380d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:59.164352 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc45b6-af09-44d7-8682-b81ed121380d admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:59.165640 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc45b6-af09-44d7-8682-b81ed121380d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:13:59.167236 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc45b6-af09-44d7-8682-b81ed121380d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:59.167950 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc45b6-af09-44d7-8682-b81ed121380d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:59.181735 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 925/1853] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:13:59 2022] PUT /identity/v3/projects/080bbbce654c4d24bf3a0bfe37ab5d24/users/800ee919d69b4d1c84618e5504ac9826/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:59.185794 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6247a4-137a-460e-a696-511abbe0dc84 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:59.188848 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 929/1854] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:13:59.194966 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc45b6-af09-44d7-8682-b81ed121380d admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:59.209889 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc45b6-af09-44d7-8682-b81ed121380d admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:59.278045 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710039d-13af-451c-8c0e-90af26edca15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:59.278681 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710039d-13af-451c-8c0e-90af26edca15 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:59.279034 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710039d-13af-451c-8c0e-90af26edca15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:59.279637 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710039d-13af-451c-8c0e-90af26edca15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:59.280533 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6710039d-13af-451c-8c0e-90af26edca15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:59.282487 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6710039d-13af-451c-8c0e-90af26edca15 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:59.308259 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6247a4-137a-460e-a696-511abbe0dc84 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:59.315086 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d727c0c-6329-4e9c-baad-cc6fa6fc97c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:59.319483 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 930/1855] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:13:59 2022] POST /identity/v3/projects => generated 375 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:13:59.323729 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710039d-13af-451c-8c0e-90af26edca15 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:59.324234 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d727c0c-6329-4e9c-baad-cc6fa6fc97c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:59.324557 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d727c0c-6329-4e9c-baad-cc6fa6fc97c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:59.325138 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d727c0c-6329-4e9c-baad-cc6fa6fc97c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:59.325972 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d727c0c-6329-4e9c-baad-cc6fa6fc97c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:59.327734 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d727c0c-6329-4e9c-baad-cc6fa6fc97c5 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:59.344305 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 926/1856] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:59.344892 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d727c0c-6329-4e9c-baad-cc6fa6fc97c5 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:59.354798 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d727c0c-6329-4e9c-baad-cc6fa6fc97c5 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:59.393343 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:59.406910 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710039d-13af-451c-8c0e-90af26edca15 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:59.413012 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d99c7b1-9f8b-4fe1-809c-7de44a9b400f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:59.414180 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99c7b1-9f8b-4fe1-809c-7de44a9b400f admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:59.414180 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99c7b1-9f8b-4fe1-809c-7de44a9b400f admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:59.414609 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99c7b1-9f8b-4fe1-809c-7de44a9b400f admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:59.415388 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d99c7b1-9f8b-4fe1-809c-7de44a9b400f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:59.417395 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d99c7b1-9f8b-4fe1-809c-7de44a9b400f admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:59.424444 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480e01e8-1c28-4e7e-8326-04da8e4cfd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:59.425131 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e01e8-1c28-4e7e-8326-04da8e4cfd10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:59.425450 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e01e8-1c28-4e7e-8326-04da8e4cfd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:59.425730 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e01e8-1c28-4e7e-8326-04da8e4cfd10 admin admin] PATH_INFO: `/v3/projects/080bbbce654c4d24bf3a0bfe37ab5d24/users/800ee919d69b4d1c84618e5504ac9826/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:59.428125 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480e01e8-1c28-4e7e-8326-04da8e4cfd10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=080bbbce654c4d24bf3a0bfe37ab5d24, user_id=800ee919d69b4d1c84618e5504ac9826, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:59.430783 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480e01e8-1c28-4e7e-8326-04da8e4cfd10 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:59.445273 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 927/1857] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:13:59 2022] PUT /identity/v3/projects/080bbbce654c4d24bf3a0bfe37ab5d24/users/800ee919d69b4d1c84618e5504ac9826/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:13:59.454016 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 931/1858] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:13:59 2022] POST /identity/v3/users => generated 288 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:13:59.456829 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480e01e8-1c28-4e7e-8326-04da8e4cfd10 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:59.464780 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c6ef8-f5db-41c1-8d3b-d00917d58f40 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:59.465183 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c6ef8-f5db-41c1-8d3b-d00917d58f40 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:59.465523 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c6ef8-f5db-41c1-8d3b-d00917d58f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:59.537498 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-bc8c6ef8-f5db-41c1-8d3b-d00917d58f40 None None] MFA Rules not processed for user `800ee919d69b4d1c84618e5504ac9826`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:59.539008 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-bc8c6ef8-f5db-41c1-8d3b-d00917d58f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:59.548584 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:59.574543 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480e01e8-1c28-4e7e-8326-04da8e4cfd10 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:59.580050 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdbf0e2-2647-4c3e-9e3a-c49ed0e5e575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:59.581267 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdbf0e2-2647-4c3e-9e3a-c49ed0e5e575 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:59.581551 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdbf0e2-2647-4c3e-9e3a-c49ed0e5e575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:59.582116 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdbf0e2-2647-4c3e-9e3a-c49ed0e5e575 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:59.582467 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdbf0e2-2647-4c3e-9e3a-c49ed0e5e575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:59.584132 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdbf0e2-2647-4c3e-9e3a-c49ed0e5e575 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:59.595385 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 928/1859] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:13:59.599260 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdbf0e2-2647-4c3e-9e3a-c49ed0e5e575 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:59.610869 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdbf0e2-2647-4c3e-9e3a-c49ed0e5e575 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:59.616529 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ba203-8ba9-4166-bf6a-6f12a635751d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:59.619118 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ba203-8ba9-4166-bf6a-6f12a635751d admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:59.619421 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ba203-8ba9-4166-bf6a-6f12a635751d admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:59.619692 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ba203-8ba9-4166-bf6a-6f12a635751d admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:59.625692 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ba203-8ba9-4166-bf6a-6f12a635751d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:59.625858 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 932/1860] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:59 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:13:59.625954 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ba203-8ba9-4166-bf6a-6f12a635751d admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:59.632510 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c6ef8-f5db-41c1-8d3b-d00917d58f40 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:59.636812 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 929/1861] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:13:59.636939 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-bc8c6ef8-f5db-41c1-8d3b-d00917d58f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:59.637890 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ba203-8ba9-4166-bf6a-6f12a635751d admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:59.649995 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ba203-8ba9-4166-bf6a-6f12a635751d admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:59.654986 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4759dd13-bc9a-40e8-9752-830da3787d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:59.656038 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4759dd13-bc9a-40e8-9752-830da3787d02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:59.656845 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4759dd13-bc9a-40e8-9752-830da3787d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:59.657124 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4759dd13-bc9a-40e8-9752-830da3787d02 admin admin] PATH_INFO: `/v3/projects/d8e5da90a062489d9f68dd6addbfd7a1/users/658aeed9964b4b33b9396eca04cffadb/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:59.675218 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4759dd13-bc9a-40e8-9752-830da3787d02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8e5da90a062489d9f68dd6addbfd7a1, user_id=658aeed9964b4b33b9396eca04cffadb, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:59.678129 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4759dd13-bc9a-40e8-9752-830da3787d02 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:59.693159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 930/1862] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:13:59 2022] PUT /identity/v3/projects/d8e5da90a062489d9f68dd6addbfd7a1/users/658aeed9964b4b33b9396eca04cffadb/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:13:59.703657 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a057eb7-0380-4335-8ab8-8317a2374f1f None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:59.704029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a057eb7-0380-4335-8ab8-8317a2374f1f None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:59.705000 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a057eb7-0380-4335-8ab8-8317a2374f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:59.707485 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c6ef8-f5db-41c1-8d3b-d00917d58f40 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:13:59.751054 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-4a057eb7-0380-4335-8ab8-8317a2374f1f None None] MFA Rules not processed for user `658aeed9964b4b33b9396eca04cffadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:13:59.753490 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-4a057eb7-0380-4335-8ab8-8317a2374f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:59.781097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf641dfd-aab9-4ca1-85a6-62c778ba3f88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:13:59.782351 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf641dfd-aab9-4ca1-85a6-62c778ba3f88 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:13:59.782749 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf641dfd-aab9-4ca1-85a6-62c778ba3f88 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:13:59.783037 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf641dfd-aab9-4ca1-85a6-62c778ba3f88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:13:59.846593 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:13:59.873255 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 931/1863] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:13:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:13:59.890525 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a057eb7-0380-4335-8ab8-8317a2374f1f None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:13:59.893117 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-4a057eb7-0380-4335-8ab8-8317a2374f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:13:59.893915 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf641dfd-aab9-4ca1-85a6-62c778ba3f88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:13:59.895628 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf641dfd-aab9-4ca1-85a6-62c778ba3f88 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:13:59.911952 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 933/1864] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:59 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:14:00.019546 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a057eb7-0380-4335-8ab8-8317a2374f1f None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:00.023796 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f7920-8cb3-4734-9c1e-2ee3399f6c63 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2092a9639067461c9a837a891f56889b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:00.024481 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f7920-8cb3-4734-9c1e-2ee3399f6c63 service glance] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:00.024691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f7920-8cb3-4734-9c1e-2ee3399f6c63 service glance] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:00.024963 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f7920-8cb3-4734-9c1e-2ee3399f6c63 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:00.038470 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f7920-8cb3-4734-9c1e-2ee3399f6c63 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:00.039777 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f7920-8cb3-4734-9c1e-2ee3399f6c63 service glance] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:00.059711 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 932/1865] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:13:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:14:00.412820 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf641dfd-aab9-4ca1-85a6-62c778ba3f88 service cinder] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:00.422205 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf641dfd-aab9-4ca1-85a6-62c778ba3f88 service cinder] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:00.436527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d8c31-c9b6-434b-9c30-6d12b4adadfa tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '800ee919d69b4d1c84618e5504ac9826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '080bbbce654c4d24bf3a0bfe37ab5d24', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:00.437073 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d8c31-c9b6-434b-9c30-6d12b4adadfa tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:00.437499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d8c31-c9b6-434b-9c30-6d12b4adadfa tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:00.438338 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d8c31-c9b6-434b-9c30-6d12b4adadfa tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:00.439022 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d8c31-c9b6-434b-9c30-6d12b4adadfa tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:14:00.439585 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d8c31-c9b6-434b-9c30-6d12b4adadfa tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:00.444672 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d8c31-c9b6-434b-9c30-6d12b4adadfa tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:00.481447 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 934/1866] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:14:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:14:00.485780 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f7920-8cb3-4734-9c1e-2ee3399f6c63 service glance] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:00.519183 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f7920-8cb3-4734-9c1e-2ee3399f6c63 service glance] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:00.538973 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3142270-41fd-483e-88f9-69bc6c8c7e21 tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '800ee919d69b4d1c84618e5504ac9826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '080bbbce654c4d24bf3a0bfe37ab5d24', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:00.539641 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3142270-41fd-483e-88f9-69bc6c8c7e21 tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:00.539949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3142270-41fd-483e-88f9-69bc6c8c7e21 tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:00.542220 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3142270-41fd-483e-88f9-69bc6c8c7e21 tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:00.543627 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3142270-41fd-483e-88f9-69bc6c8c7e21 tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:00.546025 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3142270-41fd-483e-88f9-69bc6c8c7e21 tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:00.568361 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:00.601466 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 933/1867] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:14:00 2022] POST /identity/v3/projects => generated 355 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:14:00.605435 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d8c31-c9b6-434b-9c30-6d12b4adadfa tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:00.622500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d8c31-c9b6-434b-9c30-6d12b4adadfa tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:00.627808 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3142270-41fd-483e-88f9-69bc6c8c7e21 tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:00.629665 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-d3142270-41fd-483e-88f9-69bc6c8c7e21 tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:00.731760 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c05a3a2-7965-4b9d-87fa-8176a4c2590e tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '800ee919d69b4d1c84618e5504ac9826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '080bbbce654c4d24bf3a0bfe37ab5d24', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:00.732719 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3142270-41fd-483e-88f9-69bc6c8c7e21 tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:00.733770 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05a3a2-7965-4b9d-87fa-8176a4c2590e tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:00.735767 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05a3a2-7965-4b9d-87fa-8176a4c2590e tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:00.736195 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05a3a2-7965-4b9d-87fa-8176a4c2590e tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:00.738162 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c05a3a2-7965-4b9d-87fa-8176a4c2590e tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:14:00.740645 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c05a3a2-7965-4b9d-87fa-8176a4c2590e tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:00.741076 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f277bd-b9fc-4101-ae64-c096fa7f2103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:00.741895 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f277bd-b9fc-4101-ae64-c096fa7f2103 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:00.742744 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f277bd-b9fc-4101-ae64-c096fa7f2103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:00.742744 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f277bd-b9fc-4101-ae64-c096fa7f2103 admin admin] PATH_INFO: `/v3/users/8c33fd0b6678481f99802b8d513fd0b2` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:00.744169 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f277bd-b9fc-4101-ae64-c096fa7f2103 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c33fd0b6678481f99802b8d513fd0b2)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:00.744281 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c05a3a2-7965-4b9d-87fa-8176a4c2590e tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:00.747033 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f277bd-b9fc-4101-ae64-c096fa7f2103 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:00.767290 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 935/1868] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:14:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:14:00.856859 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c05a3a2-7965-4b9d-87fa-8176a4c2590e tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:00.864052 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-54f277bd-b9fc-4101-ae64-c096fa7f2103 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c33fd0b6678481f99802b8d513fd0b2'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:00.872463 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c05a3a2-7965-4b9d-87fa-8176a4c2590e tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:00.895050 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c11e02-72fe-43f0-b3ad-5d6b1070dd78 tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '800ee919d69b4d1c84618e5504ac9826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '080bbbce654c4d24bf3a0bfe37ab5d24', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:00.895050 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c11e02-72fe-43f0-b3ad-5d6b1070dd78 tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:00.895050 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c11e02-72fe-43f0-b3ad-5d6b1070dd78 tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:00.895554 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c11e02-72fe-43f0-b3ad-5d6b1070dd78 tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] PATH_INFO: `/v3/projects/f8b29efc7c564dc992f2e13920e1feab` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:00.896823 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c11e02-72fe-43f0-b3ad-5d6b1070dd78 tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f8b29efc7c564dc992f2e13920e1feab)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:00.898653 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c11e02-72fe-43f0-b3ad-5d6b1070dd78 tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:00.904916 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-54f277bd-b9fc-4101-ae64-c096fa7f2103 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c33fd0b6678481f99802b8d513fd0b2'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:00.906017 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-54f277bd-b9fc-4101-ae64-c096fa7f2103 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c33fd0b6678481f99802b8d513fd0b2'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:00.908246 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-54f277bd-b9fc-4101-ae64-c096fa7f2103 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c33fd0b6678481f99802b8d513fd0b2'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:00.929866 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-54f277bd-b9fc-4101-ae64-c096fa7f2103 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c33fd0b6678481f99802b8d513fd0b2'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:00.966127 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-50c11e02-72fe-43f0-b3ad-5d6b1070dd78 tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8b29efc7c564dc992f2e13920e1feab', 'request_id': 'req-50c11e02-72fe-43f0-b3ad-5d6b1070dd78'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:00.980745 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-50c11e02-72fe-43f0-b3ad-5d6b1070dd78 tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8b29efc7c564dc992f2e13920e1feab', 'request_id': 'req-50c11e02-72fe-43f0-b3ad-5d6b1070dd78'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:00.988912 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-50c11e02-72fe-43f0-b3ad-5d6b1070dd78 tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8b29efc7c564dc992f2e13920e1feab', 'request_id': 'req-50c11e02-72fe-43f0-b3ad-5d6b1070dd78'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:00.992021 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 934/1869] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:14:00 2022] DELETE /identity/v3/users/8c33fd0b6678481f99802b8d513fd0b2 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:14:00.997817 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f92af9-5f3c-435c-9580-0ed1c0cae8a2 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:00.998115 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f92af9-5f3c-435c-9580-0ed1c0cae8a2 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:00.999182 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f92af9-5f3c-435c-9580-0ed1c0cae8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:01.003379 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-50c11e02-72fe-43f0-b3ad-5d6b1070dd78 tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] The token cache is being invalidate because project f8b29efc7c564dc992f2e13920e1feab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:14:01.003785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-50c11e02-72fe-43f0-b3ad-5d6b1070dd78 tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:01.004551 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-50c11e02-72fe-43f0-b3ad-5d6b1070dd78 tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:01.006152 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 936/1870] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:14:00 2022] DELETE /identity/v3/projects/f8b29efc7c564dc992f2e13920e1feab => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:14:01.036965 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-68f92af9-5f3c-435c-9580-0ed1c0cae8a2 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:01.038206 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-68f92af9-5f3c-435c-9580-0ed1c0cae8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:01.130419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 935/1871] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:01.141484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c11e02-72fe-43f0-b3ad-5d6b1070dd78 tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:01.143045 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-50c11e02-72fe-43f0-b3ad-5d6b1070dd78 tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:01.241475 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:01.259723 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c11e02-72fe-43f0-b3ad-5d6b1070dd78 tempest-VolumeQuotaClassesTest-754615387 tempest-VolumeQuotaClassesTest-754615387-project-admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:01.307612 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f92af9-5f3c-435c-9580-0ed1c0cae8a2 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:01.310483 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-68f92af9-5f3c-435c-9580-0ed1c0cae8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:01.324382 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f92af9-5f3c-435c-9580-0ed1c0cae8a2 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:01.330537 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40ec9fa-1066-4537-a235-82a8d9ff538d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:01.331373 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ec9fa-1066-4537-a235-82a8d9ff538d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:01.331636 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ec9fa-1066-4537-a235-82a8d9ff538d admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:01.332000 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ec9fa-1066-4537-a235-82a8d9ff538d admin admin] PATH_INFO: `/v3/users/8f9b4454165e4be69d3ad6481ae26a17` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:01.333208 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40ec9fa-1066-4537-a235-82a8d9ff538d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f9b4454165e4be69d3ad6481ae26a17)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:01.334391 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f801a9d-3652-49fb-a8ef-82ebca3a4a49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:01.335042 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f801a9d-3652-49fb-a8ef-82ebca3a4a49 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:01.335226 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40ec9fa-1066-4537-a235-82a8d9ff538d admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:01.335341 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f801a9d-3652-49fb-a8ef-82ebca3a4a49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:01.335598 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f801a9d-3652-49fb-a8ef-82ebca3a4a49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:01.338739 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:01.351391 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f801a9d-3652-49fb-a8ef-82ebca3a4a49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:01.353017 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f801a9d-3652-49fb-a8ef-82ebca3a4a49 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:01.377369 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 937/1872] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:14:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:14:01.417265 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-e40ec9fa-1066-4537-a235-82a8d9ff538d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f9b4454165e4be69d3ad6481ae26a17'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:01.425059 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-e40ec9fa-1066-4537-a235-82a8d9ff538d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f9b4454165e4be69d3ad6481ae26a17'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:01.425861 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-e40ec9fa-1066-4537-a235-82a8d9ff538d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f9b4454165e4be69d3ad6481ae26a17'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:01.426482 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-e40ec9fa-1066-4537-a235-82a8d9ff538d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f9b4454165e4be69d3ad6481ae26a17'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:01.443571 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-e40ec9fa-1066-4537-a235-82a8d9ff538d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f9b4454165e4be69d3ad6481ae26a17'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:01.463227 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 936/1873] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:14:01 2022] DELETE /identity/v3/users/8f9b4454165e4be69d3ad6481ae26a17 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:01.468664 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7fb53-da9a-4b12-bf76-cb79d1d8ccae None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:01.471045 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7fb53-da9a-4b12-bf76-cb79d1d8ccae None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:01.471562 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7fb53-da9a-4b12-bf76-cb79d1d8ccae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:01.500510 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-95c7fb53-da9a-4b12-bf76-cb79d1d8ccae None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:01.501732 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-95c7fb53-da9a-4b12-bf76-cb79d1d8ccae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:01.600752 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 938/1874] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:01.607588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40ec9fa-1066-4537-a235-82a8d9ff538d admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:01.609104 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-e40ec9fa-1066-4537-a235-82a8d9ff538d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:01.690072 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40ec9fa-1066-4537-a235-82a8d9ff538d admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:01.694406 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9409f454-7341-4aba-8d9a-9447c240bfae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:01.695007 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9409f454-7341-4aba-8d9a-9447c240bfae service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:01.695290 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9409f454-7341-4aba-8d9a-9447c240bfae service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:01.695671 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9409f454-7341-4aba-8d9a-9447c240bfae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:01.705943 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9409f454-7341-4aba-8d9a-9447c240bfae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:01.707482 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9409f454-7341-4aba-8d9a-9447c240bfae service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:01.718411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7fb53-da9a-4b12-bf76-cb79d1d8ccae None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:01.721085 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 937/1875] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:14:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:14:01.722481 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-95c7fb53-da9a-4b12-bf76-cb79d1d8ccae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:01.736222 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7fb53-da9a-4b12-bf76-cb79d1d8ccae None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:01.742400 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead41cdb-1f92-4bf2-a7a8-e9fbbf1c89d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:01.743109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead41cdb-1f92-4bf2-a7a8-e9fbbf1c89d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:01.743485 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead41cdb-1f92-4bf2-a7a8-e9fbbf1c89d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:01.743784 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead41cdb-1f92-4bf2-a7a8-e9fbbf1c89d5 admin admin] PATH_INFO: `/v3/projects/6afb1608b060494b8be8719ea6d77119` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:01.745058 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead41cdb-1f92-4bf2-a7a8-e9fbbf1c89d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6afb1608b060494b8be8719ea6d77119)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:01.747193 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead41cdb-1f92-4bf2-a7a8-e9fbbf1c89d5 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:01.803381 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-ead41cdb-1f92-4bf2-a7a8-e9fbbf1c89d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6afb1608b060494b8be8719ea6d77119', 'request_id': 'req-ead41cdb-1f92-4bf2-a7a8-e9fbbf1c89d5'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:01.810815 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-ead41cdb-1f92-4bf2-a7a8-e9fbbf1c89d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6afb1608b060494b8be8719ea6d77119', 'request_id': 'req-ead41cdb-1f92-4bf2-a7a8-e9fbbf1c89d5'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:01.825276 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-ead41cdb-1f92-4bf2-a7a8-e9fbbf1c89d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6afb1608b060494b8be8719ea6d77119', 'request_id': 'req-ead41cdb-1f92-4bf2-a7a8-e9fbbf1c89d5'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:01.838345 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-ead41cdb-1f92-4bf2-a7a8-e9fbbf1c89d5 admin admin] The token cache is being invalidate because project 6afb1608b060494b8be8719ea6d77119 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:14:01.838636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-ead41cdb-1f92-4bf2-a7a8-e9fbbf1c89d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:01.839452 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-ead41cdb-1f92-4bf2-a7a8-e9fbbf1c89d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:01.841031 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 939/1876] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:14:01 2022] DELETE /identity/v3/projects/6afb1608b060494b8be8719ea6d77119 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:14:01.845275 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9409f454-7341-4aba-8d9a-9447c240bfae service neutron] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:01.855812 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9409f454-7341-4aba-8d9a-9447c240bfae service neutron] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:01.975607 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df26b5-45b1-433b-b187-b9e619181520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:01.976670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df26b5-45b1-433b-b187-b9e619181520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:01.977358 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df26b5-45b1-433b-b187-b9e619181520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:01.978372 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df26b5-45b1-433b-b187-b9e619181520 admin admin] PATH_INFO: `/v3/users/af42d681a8d646b889b47e65a9dc2726` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:01.979804 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8df26b5-45b1-433b-b187-b9e619181520 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af42d681a8d646b889b47e65a9dc2726)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:01.981938 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8df26b5-45b1-433b-b187-b9e619181520 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:02.016054 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:02.063207 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-c8df26b5-45b1-433b-b187-b9e619181520 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af42d681a8d646b889b47e65a9dc2726'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:02.079476 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-c8df26b5-45b1-433b-b187-b9e619181520 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af42d681a8d646b889b47e65a9dc2726'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:02.081534 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-c8df26b5-45b1-433b-b187-b9e619181520 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af42d681a8d646b889b47e65a9dc2726'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:02.083534 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-c8df26b5-45b1-433b-b187-b9e619181520 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af42d681a8d646b889b47e65a9dc2726'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:02.118160 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-c8df26b5-45b1-433b-b187-b9e619181520 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af42d681a8d646b889b47e65a9dc2726'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:02.125177 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead41cdb-1f92-4bf2-a7a8-e9fbbf1c89d5 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:02.127333 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ead41cdb-1f92-4bf2-a7a8-e9fbbf1c89d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:02.147574 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead41cdb-1f92-4bf2-a7a8-e9fbbf1c89d5 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:02.163524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e57703-9ca4-4791-9abd-3b64f75003f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:02.164796 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e57703-9ca4-4791-9abd-3b64f75003f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:02.165021 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e57703-9ca4-4791-9abd-3b64f75003f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:02.165292 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e57703-9ca4-4791-9abd-3b64f75003f5 admin admin] PATH_INFO: `/v3/projects/bfc5015a437b403c85d9de1ba184b88d` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:02.166747 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 938/1877] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:14:01 2022] DELETE /identity/v3/users/af42d681a8d646b889b47e65a9dc2726 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:02.178001 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e57703-9ca4-4791-9abd-3b64f75003f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfc5015a437b403c85d9de1ba184b88d)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:02.180405 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e57703-9ca4-4791-9abd-3b64f75003f5 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:02.200357 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:02.253809 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-13e57703-9ca4-4791-9abd-3b64f75003f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfc5015a437b403c85d9de1ba184b88d', 'request_id': 'req-13e57703-9ca4-4791-9abd-3b64f75003f5'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:02.268538 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-13e57703-9ca4-4791-9abd-3b64f75003f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfc5015a437b403c85d9de1ba184b88d', 'request_id': 'req-13e57703-9ca4-4791-9abd-3b64f75003f5'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:02.280873 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-13e57703-9ca4-4791-9abd-3b64f75003f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfc5015a437b403c85d9de1ba184b88d', 'request_id': 'req-13e57703-9ca4-4791-9abd-3b64f75003f5'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:02.311558 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-13e57703-9ca4-4791-9abd-3b64f75003f5 admin admin] The token cache is being invalidate because project bfc5015a437b403c85d9de1ba184b88d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:14:02.311958 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-13e57703-9ca4-4791-9abd-3b64f75003f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:02.313489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-13e57703-9ca4-4791-9abd-3b64f75003f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:02.319746 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 940/1878] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:14:02 2022] DELETE /identity/v3/projects/bfc5015a437b403c85d9de1ba184b88d => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:14:02.319920 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df26b5-45b1-433b-b187-b9e619181520 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:02.321816 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-c8df26b5-45b1-433b-b187-b9e619181520 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:02.465104 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df26b5-45b1-433b-b187-b9e619181520 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:02.588954 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719ab8c-8011-4fee-a230-ff497313b633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:02.592998 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719ab8c-8011-4fee-a230-ff497313b633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:02.593429 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719ab8c-8011-4fee-a230-ff497313b633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:02.593750 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719ab8c-8011-4fee-a230-ff497313b633 admin admin] PATH_INFO: `/v3/users/800ee919d69b4d1c84618e5504ac9826` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:02.596657 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e719ab8c-8011-4fee-a230-ff497313b633 admin admin] RBAC: Authorizing `identity:delete_user(user_id=800ee919d69b4d1c84618e5504ac9826)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:02.601043 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e719ab8c-8011-4fee-a230-ff497313b633 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:02.683778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-e719ab8c-8011-4fee-a230-ff497313b633 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '800ee919d69b4d1c84618e5504ac9826'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:02.689709 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:02.708524 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-e719ab8c-8011-4fee-a230-ff497313b633 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '800ee919d69b4d1c84618e5504ac9826'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:02.709504 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-e719ab8c-8011-4fee-a230-ff497313b633 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '800ee919d69b4d1c84618e5504ac9826'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:02.710610 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-e719ab8c-8011-4fee-a230-ff497313b633 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '800ee919d69b4d1c84618e5504ac9826'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:02.731474 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-e719ab8c-8011-4fee-a230-ff497313b633 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '800ee919d69b4d1c84618e5504ac9826'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:02.778050 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 939/1879] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:14:02 2022] DELETE /identity/v3/users/800ee919d69b4d1c84618e5504ac9826 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:14:02.910592 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e57703-9ca4-4791-9abd-3b64f75003f5 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:02.912704 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-13e57703-9ca4-4791-9abd-3b64f75003f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:02.931936 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e57703-9ca4-4791-9abd-3b64f75003f5 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:03.105881 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b656f1-cfef-41d7-8c8c-b297369ec78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:03.108038 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b656f1-cfef-41d7-8c8c-b297369ec78a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:03.108325 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b656f1-cfef-41d7-8c8c-b297369ec78a admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:03.108693 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b656f1-cfef-41d7-8c8c-b297369ec78a admin admin] PATH_INFO: `/v3/projects/47bea51bd78b4b618c68c32b9c5341dc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:03.114955 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b656f1-cfef-41d7-8c8c-b297369ec78a admin admin] RBAC: Authorizing `identity:delete_project(project_id=47bea51bd78b4b618c68c32b9c5341dc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:03.117764 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b656f1-cfef-41d7-8c8c-b297369ec78a admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:03.196849 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f8b656f1-cfef-41d7-8c8c-b297369ec78a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47bea51bd78b4b618c68c32b9c5341dc', 'request_id': 'req-f8b656f1-cfef-41d7-8c8c-b297369ec78a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:03.219535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f8b656f1-cfef-41d7-8c8c-b297369ec78a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47bea51bd78b4b618c68c32b9c5341dc', 'request_id': 'req-f8b656f1-cfef-41d7-8c8c-b297369ec78a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:03.229239 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f8b656f1-cfef-41d7-8c8c-b297369ec78a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47bea51bd78b4b618c68c32b9c5341dc', 'request_id': 'req-f8b656f1-cfef-41d7-8c8c-b297369ec78a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:03.249436 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f8b656f1-cfef-41d7-8c8c-b297369ec78a admin admin] The token cache is being invalidate because project 47bea51bd78b4b618c68c32b9c5341dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:14:03.249846 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f8b656f1-cfef-41d7-8c8c-b297369ec78a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:03.250601 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f8b656f1-cfef-41d7-8c8c-b297369ec78a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:03.253419 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 941/1880] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:14:02 2022] DELETE /identity/v3/projects/47bea51bd78b4b618c68c32b9c5341dc => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:03.259777 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c7661-ec0c-49e2-90cc-565d7174ee72 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:03.260060 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c7661-ec0c-49e2-90cc-565d7174ee72 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:03.260330 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c7661-ec0c-49e2-90cc-565d7174ee72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:03.291197 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-9e9c7661-ec0c-49e2-90cc-565d7174ee72 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:03.292967 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-9e9c7661-ec0c-49e2-90cc-565d7174ee72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:03.312632 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b656f1-cfef-41d7-8c8c-b297369ec78a admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:03.314826 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f8b656f1-cfef-41d7-8c8c-b297369ec78a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:03.369603 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:03.403067 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 940/1881] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:03.404523 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacd8c5-e879-43a7-9935-8d371420b8d1 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:03.404821 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacd8c5-e879-43a7-9935-8d371420b8d1 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:03.405091 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacd8c5-e879-43a7-9935-8d371420b8d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:03.413484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b656f1-cfef-41d7-8c8c-b297369ec78a admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:03.420372 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c6f48-294c-4493-acf0-f8db0ea44027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:03.421245 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c6f48-294c-4493-acf0-f8db0ea44027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:03.421523 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c6f48-294c-4493-acf0-f8db0ea44027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:03.421855 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c6f48-294c-4493-acf0-f8db0ea44027 admin admin] PATH_INFO: `/v3/projects/080bbbce654c4d24bf3a0bfe37ab5d24` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:03.424083 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c6f48-294c-4493-acf0-f8db0ea44027 admin admin] RBAC: Authorizing `identity:delete_project(project_id=080bbbce654c4d24bf3a0bfe37ab5d24)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:03.426013 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c6f48-294c-4493-acf0-f8db0ea44027 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:03.428738 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-6bacd8c5-e879-43a7-9935-8d371420b8d1 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:03.429953 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-6bacd8c5-e879-43a7-9935-8d371420b8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:03.439034 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 941/1882] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:03.442639 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bacd8c5-e879-43a7-9935-8d371420b8d1 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:03.453903 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bacd8c5-e879-43a7-9935-8d371420b8d1 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:03.459378 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa8ddab-cf1c-4045-a260-1176d9415348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:03.460067 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8ddab-cf1c-4045-a260-1176d9415348 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:03.460523 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8ddab-cf1c-4045-a260-1176d9415348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:03.460808 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8ddab-cf1c-4045-a260-1176d9415348 admin admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:03.461448 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa8ddab-cf1c-4045-a260-1176d9415348 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:14:03.461811 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa8ddab-cf1c-4045-a260-1176d9415348 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:03.464063 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa8ddab-cf1c-4045-a260-1176d9415348 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:03.479681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 942/1883] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:14:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:14:03.480737 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa8ddab-cf1c-4045-a260-1176d9415348 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:03.491123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-372c6f48-294c-4493-acf0-f8db0ea44027 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '080bbbce654c4d24bf3a0bfe37ab5d24', 'request_id': 'req-372c6f48-294c-4493-acf0-f8db0ea44027'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:03.495299 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa8ddab-cf1c-4045-a260-1176d9415348 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:03.500848 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-372c6f48-294c-4493-acf0-f8db0ea44027 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '080bbbce654c4d24bf3a0bfe37ab5d24', 'request_id': 'req-372c6f48-294c-4493-acf0-f8db0ea44027'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:03.509072 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-372c6f48-294c-4493-acf0-f8db0ea44027 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '080bbbce654c4d24bf3a0bfe37ab5d24', 'request_id': 'req-372c6f48-294c-4493-acf0-f8db0ea44027'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:03.522063 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-372c6f48-294c-4493-acf0-f8db0ea44027 admin admin] The token cache is being invalidate because project 080bbbce654c4d24bf3a0bfe37ab5d24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:14:03.522353 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-372c6f48-294c-4493-acf0-f8db0ea44027 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:03.523197 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-372c6f48-294c-4493-acf0-f8db0ea44027 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:03.528304 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 942/1884] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:14:03 2022] DELETE /identity/v3/projects/080bbbce654c4d24bf3a0bfe37ab5d24 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:14:03.530408 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34c3eb-823c-4d6e-a6f6-c75ad1899ca1 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:03.530700 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34c3eb-823c-4d6e-a6f6-c75ad1899ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:03.531005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34c3eb-823c-4d6e-a6f6-c75ad1899ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:03.551866 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-0b34c3eb-823c-4d6e-a6f6-c75ad1899ca1 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:03.553122 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-0b34c3eb-823c-4d6e-a6f6-c75ad1899ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:03.569895 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed31594-397a-4fe9-8d5f-caa5c2d9d785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:03.570520 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed31594-397a-4fe9-8d5f-caa5c2d9d785 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:03.570800 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed31594-397a-4fe9-8d5f-caa5c2d9d785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:03.571103 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed31594-397a-4fe9-8d5f-caa5c2d9d785 admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:03.571741 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed31594-397a-4fe9-8d5f-caa5c2d9d785 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:03.573665 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed31594-397a-4fe9-8d5f-caa5c2d9d785 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:03.609503 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 943/1885] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:14:03 2022] POST /identity/v3/projects => generated 409 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:14:03.613975 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed31594-397a-4fe9-8d5f-caa5c2d9d785 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:03.616014 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-8ed31594-397a-4fe9-8d5f-caa5c2d9d785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:03.620819 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 943/1886] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:03.629219 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19025410-0143-4e5b-9bba-648e8152fb6e None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:03.629607 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19025410-0143-4e5b-9bba-648e8152fb6e None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:03.629973 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19025410-0143-4e5b-9bba-648e8152fb6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:03.650510 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-19025410-0143-4e5b-9bba-648e8152fb6e None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:03.651857 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-19025410-0143-4e5b-9bba-648e8152fb6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:03.659484 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:03.688093 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed31594-397a-4fe9-8d5f-caa5c2d9d785 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:03.694683 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f93e534-9ced-428d-aa64-6d805504962f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:03.695350 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93e534-9ced-428d-aa64-6d805504962f admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:03.695632 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93e534-9ced-428d-aa64-6d805504962f admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:03.695905 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93e534-9ced-428d-aa64-6d805504962f admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:03.696544 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f93e534-9ced-428d-aa64-6d805504962f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:03.698860 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f93e534-9ced-428d-aa64-6d805504962f admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:03.717635 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 944/1887] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:14:03.723154 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19025410-0143-4e5b-9bba-648e8152fb6e None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:03.728275 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 944/1888] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:14:03 2022] POST /identity/v3/users => generated 305 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:14:03.729432 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:03.732522 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f93e534-9ced-428d-aa64-6d805504962f admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:03.737245 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19025410-0143-4e5b-9bba-648e8152fb6e None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:03.744855 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b19f8-be95-4d0e-ae88-d2c4168bd499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:03.746032 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b19f8-be95-4d0e-ae88-d2c4168bd499 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:03.746313 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b19f8-be95-4d0e-ae88-d2c4168bd499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:03.746586 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b19f8-be95-4d0e-ae88-d2c4168bd499 admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:03.747828 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b19f8-be95-4d0e-ae88-d2c4168bd499 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:14:03.748743 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b19f8-be95-4d0e-ae88-d2c4168bd499 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:03.749995 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b19f8-be95-4d0e-ae88-d2c4168bd499 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:03.751506 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f93e534-9ced-428d-aa64-6d805504962f admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:03.756764 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4df6a0-1bdf-4e10-86f8-b254e6b3067a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:03.757464 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4df6a0-1bdf-4e10-86f8-b254e6b3067a admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:03.757745 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4df6a0-1bdf-4e10-86f8-b254e6b3067a admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:03.758419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4df6a0-1bdf-4e10-86f8-b254e6b3067a admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:03.758991 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4df6a0-1bdf-4e10-86f8-b254e6b3067a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:03.761609 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4df6a0-1bdf-4e10-86f8-b254e6b3067a admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:03.764606 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 945/1889] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:14:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:14:03.769912 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b19f8-be95-4d0e-ae88-d2c4168bd499 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:03.785409 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 945/1890] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:14:03.787040 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b19f8-be95-4d0e-ae88-d2c4168bd499 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:03.789176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4df6a0-1bdf-4e10-86f8-b254e6b3067a admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:03.791815 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ef187-8038-47e5-b390-4161eacd178a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:03.792477 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ef187-8038-47e5-b390-4161eacd178a admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:03.792840 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ef187-8038-47e5-b390-4161eacd178a admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:03.793143 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ef187-8038-47e5-b390-4161eacd178a admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:03.793886 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ef187-8038-47e5-b390-4161eacd178a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:03.795834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ef187-8038-47e5-b390-4161eacd178a admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:03.811415 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4df6a0-1bdf-4e10-86f8-b254e6b3067a admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:03.836772 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5140b176-318e-45d8-8887-9ac79ff64c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:03.837527 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5140b176-318e-45d8-8887-9ac79ff64c74 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:03.837795 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5140b176-318e-45d8-8887-9ac79ff64c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:03.838067 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5140b176-318e-45d8-8887-9ac79ff64c74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:03.838712 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5140b176-318e-45d8-8887-9ac79ff64c74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:03.840354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5140b176-318e-45d8-8887-9ac79ff64c74 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:03.856695 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 946/1891] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:14:03.858010 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5140b176-318e-45d8-8887-9ac79ff64c74 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:03.866768 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 946/1892] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:14:03 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:14:03.869926 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ef187-8038-47e5-b390-4161eacd178a admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:03.871897 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5140b176-318e-45d8-8887-9ac79ff64c74 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:03.982120 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2b64e9-6fb6-45aa-ab07-4a714d51b39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:03.984534 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ef187-8038-47e5-b390-4161eacd178a admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:03.986191 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2b64e9-6fb6-45aa-ab07-4a714d51b39a admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:03.986797 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2b64e9-6fb6-45aa-ab07-4a714d51b39a admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:03.987585 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2b64e9-6fb6-45aa-ab07-4a714d51b39a admin admin] PATH_INFO: `/v3/projects/0f9f744449744d959abd33704b09fac8/users/a2357ca63bff41509e18ed10d9b2214c/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:03.992132 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff11e4-95c0-4ea1-9a89-6f08772b4cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:03.993358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff11e4-95c0-4ea1-9a89-6f08772b4cdb admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:03.993657 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff11e4-95c0-4ea1-9a89-6f08772b4cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:03.993995 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff11e4-95c0-4ea1-9a89-6f08772b4cdb admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:03.995157 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ff11e4-95c0-4ea1-9a89-6f08772b4cdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:03.997857 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ff11e4-95c0-4ea1-9a89-6f08772b4cdb admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:04.011042 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2b64e9-6fb6-45aa-ab07-4a714d51b39a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f9f744449744d959abd33704b09fac8, user_id=a2357ca63bff41509e18ed10d9b2214c, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:04.032446 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2b64e9-6fb6-45aa-ab07-4a714d51b39a admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:04.054905 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 947/1893] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:14:03 2022] POST /identity/v3/users => generated 291 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:14:04.066903 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 947/1894] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:14:03 2022] PUT /identity/v3/projects/0f9f744449744d959abd33704b09fac8/users/a2357ca63bff41509e18ed10d9b2214c/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:04.067258 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff11e4-95c0-4ea1-9a89-6f08772b4cdb admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:04.075891 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e73ab-28b5-47c5-8b9e-ed497d746d73 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:04.076194 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e73ab-28b5-47c5-8b9e-ed497d746d73 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:04.076464 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e73ab-28b5-47c5-8b9e-ed497d746d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:04.092420 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff11e4-95c0-4ea1-9a89-6f08772b4cdb admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:04.155889 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-b12e73ab-28b5-47c5-8b9e-ed497d746d73 None None] MFA Rules not processed for user `a2357ca63bff41509e18ed10d9b2214c`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:04.155889 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-b12e73ab-28b5-47c5-8b9e-ed497d746d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:04.250641 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f390d-e814-4bb1-8b85-29c42dbf9de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:04.252358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f390d-e814-4bb1-8b85-29c42dbf9de4 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:04.253059 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f390d-e814-4bb1-8b85-29c42dbf9de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:04.253543 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f390d-e814-4bb1-8b85-29c42dbf9de4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:04.254600 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85f390d-e814-4bb1-8b85-29c42dbf9de4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:04.265023 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85f390d-e814-4bb1-8b85-29c42dbf9de4 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:04.304573 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 948/1895] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:14:04.309917 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f390d-e814-4bb1-8b85-29c42dbf9de4 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:04.316267 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 948/1896] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:04 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:14:04.320276 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e73ab-28b5-47c5-8b9e-ed497d746d73 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:04.333891 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f390d-e814-4bb1-8b85-29c42dbf9de4 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:04.335582 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e73ab-28b5-47c5-8b9e-ed497d746d73 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:04.343939 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0af3d9-c3ff-4842-b2e0-b54d87b0b3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:04.344097 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ca1a9a-2555-4d17-bf24-f609ee2c79af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:04.344215 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0af3d9-c3ff-4842-b2e0-b54d87b0b3f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:04.344292 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca1a9a-2555-4d17-bf24-f609ee2c79af admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:04.344493 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0af3d9-c3ff-4842-b2e0-b54d87b0b3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:04.344586 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca1a9a-2555-4d17-bf24-f609ee2c79af admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:04.344667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0af3d9-c3ff-4842-b2e0-b54d87b0b3f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:04.344850 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca1a9a-2555-4d17-bf24-f609ee2c79af admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:04.345505 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0af3d9-c3ff-4842-b2e0-b54d87b0b3f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:04.345609 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ca1a9a-2555-4d17-bf24-f609ee2c79af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:04.346907 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0af3d9-c3ff-4842-b2e0-b54d87b0b3f6 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:04.347523 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ca1a9a-2555-4d17-bf24-f609ee2c79af admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:04.359780 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 949/1897] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:14:04.363519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0af3d9-c3ff-4842-b2e0-b54d87b0b3f6 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:04.378385 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:04.380915 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0af3d9-c3ff-4842-b2e0-b54d87b0b3f6 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:04.387870 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082be263-6b74-4c8d-a8a1-68a5a59c762d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:04.388592 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082be263-6b74-4c8d-a8a1-68a5a59c762d admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:04.388913 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082be263-6b74-4c8d-a8a1-68a5a59c762d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:04.389311 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082be263-6b74-4c8d-a8a1-68a5a59c762d admin admin] PATH_INFO: `/v3/projects/7fb787f6e0a8414787a501e06bdfba6f/users/0fdd7c465f654d0fad432822b6c42ffa/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:04.404290 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 949/1898] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:14:04 2022] POST /identity/v3/projects => generated 409 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:14:04.408985 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ca1a9a-2555-4d17-bf24-f609ee2c79af admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:04.411778 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082be263-6b74-4c8d-a8a1-68a5a59c762d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fb787f6e0a8414787a501e06bdfba6f, user_id=0fdd7c465f654d0fad432822b6c42ffa, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:04.415107 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082be263-6b74-4c8d-a8a1-68a5a59c762d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:04.431747 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 950/1899] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:14:04 2022] PUT /identity/v3/projects/7fb787f6e0a8414787a501e06bdfba6f/users/0fdd7c465f654d0fad432822b6c42ffa/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:14:04.443769 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe5c1f-6cd0-4549-a94b-1f5420287423 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:04.444148 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe5c1f-6cd0-4549-a94b-1f5420287423 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:04.444770 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe5c1f-6cd0-4549-a94b-1f5420287423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:04.479110 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-6abe5c1f-6cd0-4549-a94b-1f5420287423 None None] MFA Rules not processed for user `0fdd7c465f654d0fad432822b6c42ffa`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:04.480246 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-6abe5c1f-6cd0-4549-a94b-1f5420287423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:04.490794 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:04.495256 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ca1a9a-2555-4d17-bf24-f609ee2c79af admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:04.558751 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ceec30-0925-48b7-8ca1-30cfacea2fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:04.559425 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ceec30-0925-48b7-8ca1-30cfacea2fd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:04.559666 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ceec30-0925-48b7-8ca1-30cfacea2fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:04.559905 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 951/1900] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:14:04.560003 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ceec30-0925-48b7-8ca1-30cfacea2fd5 admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:04.560564 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ceec30-0925-48b7-8ca1-30cfacea2fd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:04.563077 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ceec30-0925-48b7-8ca1-30cfacea2fd5 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:04.568293 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe5c1f-6cd0-4549-a94b-1f5420287423 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:04.580427 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe5c1f-6cd0-4549-a94b-1f5420287423 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:04.585549 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78516aa1-dbee-4d69-85d8-390c43140b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:04.586283 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78516aa1-dbee-4d69-85d8-390c43140b6d admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:04.586575 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78516aa1-dbee-4d69-85d8-390c43140b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:04.586827 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78516aa1-dbee-4d69-85d8-390c43140b6d admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:04.587560 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78516aa1-dbee-4d69-85d8-390c43140b6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:04.592505 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78516aa1-dbee-4d69-85d8-390c43140b6d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:04.592708 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 950/1901] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:14:04 2022] POST /identity/v3/users => generated 311 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:14:04.592708 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ceec30-0925-48b7-8ca1-30cfacea2fd5 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:04.603351 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ceec30-0925-48b7-8ca1-30cfacea2fd5 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:04.610922 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9080266-4149-475e-96d3-53c466aacdef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:04.611587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9080266-4149-475e-96d3-53c466aacdef admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:04.611798 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9080266-4149-475e-96d3-53c466aacdef admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:04.612060 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9080266-4149-475e-96d3-53c466aacdef admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:04.612702 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9080266-4149-475e-96d3-53c466aacdef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:04.614540 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9080266-4149-475e-96d3-53c466aacdef admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:04.626506 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 951/1902] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:14:04.627333 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 952/1903] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:14:04 2022] POST /identity/v3/projects => generated 381 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:14:04.630861 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9080266-4149-475e-96d3-53c466aacdef admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:04.631911 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78516aa1-dbee-4d69-85d8-390c43140b6d admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:04.647128 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9080266-4149-475e-96d3-53c466aacdef admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:04.746870 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c051cf6c-8ac3-467c-9a9b-024eb6e00730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:04.747535 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051cf6c-8ac3-467c-9a9b-024eb6e00730 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:04.747804 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051cf6c-8ac3-467c-9a9b-024eb6e00730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:04.748475 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051cf6c-8ac3-467c-9a9b-024eb6e00730 admin admin] PATH_INFO: `/v3/projects/1a7398341fec445ab429c299872c1ab5/users/5ef624a43d9b4d87970e896212202f5c/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:04.750785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78516aa1-dbee-4d69-85d8-390c43140b6d admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:04.757110 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0cf9c0-22a1-4b84-b622-ebd62c06bfcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:04.757833 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0cf9c0-22a1-4b84-b622-ebd62c06bfcf admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:04.759046 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0cf9c0-22a1-4b84-b622-ebd62c06bfcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:04.759046 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0cf9c0-22a1-4b84-b622-ebd62c06bfcf admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:04.761594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0cf9c0-22a1-4b84-b622-ebd62c06bfcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:04.761972 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0cf9c0-22a1-4b84-b622-ebd62c06bfcf admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:04.778150 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c051cf6c-8ac3-467c-9a9b-024eb6e00730 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a7398341fec445ab429c299872c1ab5, user_id=5ef624a43d9b4d87970e896212202f5c, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:04.786190 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c051cf6c-8ac3-467c-9a9b-024eb6e00730 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:04.794159 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 953/1904] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:14:04 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:14:04.800090 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0cf9c0-22a1-4b84-b622-ebd62c06bfcf admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:04.820645 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 952/1905] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:14:04 2022] PUT /identity/v3/projects/1a7398341fec445ab429c299872c1ab5/users/5ef624a43d9b4d87970e896212202f5c/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:14:04.824353 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c051cf6c-8ac3-467c-9a9b-024eb6e00730 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:04.830082 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:04.831813 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0cf9c0-22a1-4b84-b622-ebd62c06bfcf admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:04.919500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7148ca3-8278-4627-a178-7cbdf04fe898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:04.920131 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7148ca3-8278-4627-a178-7cbdf04fe898 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:04.920415 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7148ca3-8278-4627-a178-7cbdf04fe898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:04.920690 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7148ca3-8278-4627-a178-7cbdf04fe898 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:04.921346 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7148ca3-8278-4627-a178-7cbdf04fe898 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:04.923567 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7148ca3-8278-4627-a178-7cbdf04fe898 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:04.929567 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c051cf6c-8ac3-467c-9a9b-024eb6e00730 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:04.934858 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 954/1906] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:14:04.935596 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f298cde-e25d-4ad3-abbb-41e7a8765c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:04.936217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f298cde-e25d-4ad3-abbb-41e7a8765c23 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:04.936505 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f298cde-e25d-4ad3-abbb-41e7a8765c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:04.936769 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f298cde-e25d-4ad3-abbb-41e7a8765c23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:04.937444 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f298cde-e25d-4ad3-abbb-41e7a8765c23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:04.939131 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f298cde-e25d-4ad3-abbb-41e7a8765c23 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:04.940207 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7148ca3-8278-4627-a178-7cbdf04fe898 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:04.951109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7148ca3-8278-4627-a178-7cbdf04fe898 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:04.951857 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 953/1907] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:14:04.955900 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f298cde-e25d-4ad3-abbb-41e7a8765c23 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:04.959455 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf57149-6ddb-4ccf-b4f7-dd7848036ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:04.960125 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf57149-6ddb-4ccf-b4f7-dd7848036ad6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:04.960465 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf57149-6ddb-4ccf-b4f7-dd7848036ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:04.960743 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf57149-6ddb-4ccf-b4f7-dd7848036ad6 admin admin] PATH_INFO: `/v3/projects/094ca5d83ac04bb38d799646353515f9/users/3e2c2bdaa94a4b64aed0277d694f8cdc/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:04.974729 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:04.979283 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f298cde-e25d-4ad3-abbb-41e7a8765c23 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:04.985160 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb05b3e-c2d1-40de-83fc-5e1f252c8eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:04.985439 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf57149-6ddb-4ccf-b4f7-dd7848036ad6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=094ca5d83ac04bb38d799646353515f9, user_id=3e2c2bdaa94a4b64aed0277d694f8cdc, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:04.986070 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb05b3e-c2d1-40de-83fc-5e1f252c8eb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:04.986178 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb05b3e-c2d1-40de-83fc-5e1f252c8eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:04.986471 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb05b3e-c2d1-40de-83fc-5e1f252c8eb5 admin admin] PATH_INFO: `/v3/projects/1a7398341fec445ab429c299872c1ab5/users/5ef624a43d9b4d87970e896212202f5c/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:04.988147 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf57149-6ddb-4ccf-b4f7-dd7848036ad6 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:04.988621 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb05b3e-c2d1-40de-83fc-5e1f252c8eb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a7398341fec445ab429c299872c1ab5, user_id=5ef624a43d9b4d87970e896212202f5c, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:04.991229 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb05b3e-c2d1-40de-83fc-5e1f252c8eb5 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:05.003255 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 954/1908] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:14:04 2022] PUT /identity/v3/projects/1a7398341fec445ab429c299872c1ab5/users/5ef624a43d9b4d87970e896212202f5c/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:05.004789 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 955/1909] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:14:04 2022] PUT /identity/v3/projects/094ca5d83ac04bb38d799646353515f9/users/3e2c2bdaa94a4b64aed0277d694f8cdc/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:05.012600 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb05b3e-c2d1-40de-83fc-5e1f252c8eb5 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:05.013335 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba759fac-5d27-4f03-a120-3cbf0c974892 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:05.013752 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba759fac-5d27-4f03-a120-3cbf0c974892 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:05.014082 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba759fac-5d27-4f03-a120-3cbf0c974892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:05.064581 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-ba759fac-5d27-4f03-a120-3cbf0c974892 None None] MFA Rules not processed for user `5ef624a43d9b4d87970e896212202f5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:05.065908 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ba759fac-5d27-4f03-a120-3cbf0c974892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:05.130465 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb05b3e-c2d1-40de-83fc-5e1f252c8eb5 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:05.223324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376d5b8-65f2-412b-9c27-a431531a2216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:05.223940 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376d5b8-65f2-412b-9c27-a431531a2216 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:05.224214 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376d5b8-65f2-412b-9c27-a431531a2216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:05.224482 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376d5b8-65f2-412b-9c27-a431531a2216 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:05.225125 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3376d5b8-65f2-412b-9c27-a431531a2216 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:05.226710 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3376d5b8-65f2-412b-9c27-a431531a2216 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:05.237987 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 955/1910] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:05 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:14:05.242394 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376d5b8-65f2-412b-9c27-a431531a2216 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:05.245309 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 956/1911] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:05 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:05.253940 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376d5b8-65f2-412b-9c27-a431531a2216 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:05.259515 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e43eb4-0885-4540-ba48-0c2757aa5fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:05.260561 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e43eb4-0885-4540-ba48-0c2757aa5fd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:05.260977 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e43eb4-0885-4540-ba48-0c2757aa5fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:05.261320 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e43eb4-0885-4540-ba48-0c2757aa5fd7 admin admin] PATH_INFO: `/v3/projects/094ca5d83ac04bb38d799646353515f9/users/3e2c2bdaa94a4b64aed0277d694f8cdc/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:05.262240 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba759fac-5d27-4f03-a120-3cbf0c974892 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:05.263438 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e43eb4-0885-4540-ba48-0c2757aa5fd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=094ca5d83ac04bb38d799646353515f9, user_id=3e2c2bdaa94a4b64aed0277d694f8cdc, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:05.265308 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ba759fac-5d27-4f03-a120-3cbf0c974892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:05.266886 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e43eb4-0885-4540-ba48-0c2757aa5fd7 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:05.280073 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 956/1912] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:14:05 2022] PUT /identity/v3/projects/094ca5d83ac04bb38d799646353515f9/users/3e2c2bdaa94a4b64aed0277d694f8cdc/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:14:05.292468 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f3832-a922-46d4-be08-96ec9e09b297 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:05.292746 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f3832-a922-46d4-be08-96ec9e09b297 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:05.293534 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f3832-a922-46d4-be08-96ec9e09b297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:05.326137 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-e38f3832-a922-46d4-be08-96ec9e09b297 None None] MFA Rules not processed for user `3e2c2bdaa94a4b64aed0277d694f8cdc`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:05.327475 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-e38f3832-a922-46d4-be08-96ec9e09b297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:05.332537 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:05.373860 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba759fac-5d27-4f03-a120-3cbf0c974892 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:05.421444 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 957/1913] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:05 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:05.431504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe691248-daf5-4069-a3bb-adaf19ddb7a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:05.432127 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe691248-daf5-4069-a3bb-adaf19ddb7a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:05.432408 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe691248-daf5-4069-a3bb-adaf19ddb7a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:05.432682 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe691248-daf5-4069-a3bb-adaf19ddb7a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:05.432857 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f3832-a922-46d4-be08-96ec9e09b297 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:05.434331 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-e38f3832-a922-46d4-be08-96ec9e09b297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:05.455744 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f3832-a922-46d4-be08-96ec9e09b297 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:05.463281 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8639f70f-acfb-4a50-a156-2444d7da30f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:05.464217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639f70f-acfb-4a50-a156-2444d7da30f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:05.464852 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639f70f-acfb-4a50-a156-2444d7da30f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:05.465483 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639f70f-acfb-4a50-a156-2444d7da30f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:05.481165 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8639f70f-acfb-4a50-a156-2444d7da30f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:05.483070 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8639f70f-acfb-4a50-a156-2444d7da30f0 service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:05.497264 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 958/1914] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:14:05 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:14:05.498767 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:05.531856 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe691248-daf5-4069-a3bb-adaf19ddb7a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:05.533514 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe691248-daf5-4069-a3bb-adaf19ddb7a8 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:05.556796 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 957/1915] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:14:05 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:14:05.755011 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8639f70f-acfb-4a50-a156-2444d7da30f0 service cinder] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:05.767061 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8639f70f-acfb-4a50-a156-2444d7da30f0 service cinder] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:05.773356 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677bf10e-46b7-4d92-a965-023808f703fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:05.774105 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677bf10e-46b7-4d92-a965-023808f703fe service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:05.774389 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677bf10e-46b7-4d92-a965-023808f703fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:05.774669 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677bf10e-46b7-4d92-a965-023808f703fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:05.821916 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe691248-daf5-4069-a3bb-adaf19ddb7a8 service cinder] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:05.841343 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe691248-daf5-4069-a3bb-adaf19ddb7a8 service cinder] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:05.846604 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b379e66-38a4-4451-a954-bbeace360885 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:05.847373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b379e66-38a4-4451-a954-bbeace360885 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:05.847752 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b379e66-38a4-4451-a954-bbeace360885 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:05.848736 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b379e66-38a4-4451-a954-bbeace360885 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:05.865512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677bf10e-46b7-4d92-a965-023808f703fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:05.867029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677bf10e-46b7-4d92-a965-023808f703fe service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:05.891631 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 959/1916] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:14:05 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:14:05.941236 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b379e66-38a4-4451-a954-bbeace360885 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:05.943129 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b379e66-38a4-4451-a954-bbeace360885 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:05.957320 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 958/1917] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:14:05 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:14:12.340156 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677bf10e-46b7-4d92-a965-023808f703fe service cinder] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:12.463203 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677bf10e-46b7-4d92-a965-023808f703fe service cinder] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:12.468086 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8617dd07-6775-4774-8f96-8d9f7663046c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:12.468858 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617dd07-6775-4774-8f96-8d9f7663046c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:12.469201 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617dd07-6775-4774-8f96-8d9f7663046c admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:12.469539 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617dd07-6775-4774-8f96-8d9f7663046c admin admin] PATH_INFO: `/v3/users/0fdd7c465f654d0fad432822b6c42ffa` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:12.470985 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8617dd07-6775-4774-8f96-8d9f7663046c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fdd7c465f654d0fad432822b6c42ffa)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:12.473065 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8617dd07-6775-4774-8f96-8d9f7663046c admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:12.517134 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:12.558022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-8617dd07-6775-4774-8f96-8d9f7663046c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fdd7c465f654d0fad432822b6c42ffa'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:12.576612 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-8617dd07-6775-4774-8f96-8d9f7663046c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fdd7c465f654d0fad432822b6c42ffa'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:12.577563 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-8617dd07-6775-4774-8f96-8d9f7663046c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fdd7c465f654d0fad432822b6c42ffa'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:12.579043 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-8617dd07-6775-4774-8f96-8d9f7663046c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fdd7c465f654d0fad432822b6c42ffa'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:12.608703 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-8617dd07-6775-4774-8f96-8d9f7663046c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fdd7c465f654d0fad432822b6c42ffa'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:12.641905 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 960/1918] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:14:12 2022] DELETE /identity/v3/users/0fdd7c465f654d0fad432822b6c42ffa => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:12.643314 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a029b-4c53-4893-aa47-ac32df3f124c None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:12.643779 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a029b-4c53-4893-aa47-ac32df3f124c None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:12.643880 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a029b-4c53-4893-aa47-ac32df3f124c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:12.696073 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-391a029b-4c53-4893-aa47-ac32df3f124c None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:12.698795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-391a029b-4c53-4893-aa47-ac32df3f124c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:12.819538 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 959/1919] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:14:12.853060 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8617dd07-6775-4774-8f96-8d9f7663046c admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:12.854581 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-8617dd07-6775-4774-8f96-8d9f7663046c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:12.995570 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8617dd07-6775-4774-8f96-8d9f7663046c admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:13.000321 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb40bf-3f9a-4e55-b727-b58cf7060944 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:13.000957 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb40bf-3f9a-4e55-b727-b58cf7060944 service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:13.001234 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb40bf-3f9a-4e55-b727-b58cf7060944 service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:13.001506 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb40bf-3f9a-4e55-b727-b58cf7060944 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:13.011979 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eb40bf-3f9a-4e55-b727-b58cf7060944 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:13.019772 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eb40bf-3f9a-4e55-b727-b58cf7060944 service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:13.031854 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 961/1920] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:14:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:14:13.410979 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a029b-4c53-4893-aa47-ac32df3f124c None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:13.412415 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-391a029b-4c53-4893-aa47-ac32df3f124c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:13.425383 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a029b-4c53-4893-aa47-ac32df3f124c None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:13.430100 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea60d90-3667-43e5-86b4-470524cd5434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:13.430773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea60d90-3667-43e5-86b4-470524cd5434 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:13.431151 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea60d90-3667-43e5-86b4-470524cd5434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:13.439405 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea60d90-3667-43e5-86b4-470524cd5434 admin admin] PATH_INFO: `/v3/projects/7fb787f6e0a8414787a501e06bdfba6f` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:13.440903 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea60d90-3667-43e5-86b4-470524cd5434 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fb787f6e0a8414787a501e06bdfba6f)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:13.442966 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea60d90-3667-43e5-86b4-470524cd5434 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:13.495643 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-fea60d90-3667-43e5-86b4-470524cd5434 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fb787f6e0a8414787a501e06bdfba6f', 'request_id': 'req-fea60d90-3667-43e5-86b4-470524cd5434'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:13.504167 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-fea60d90-3667-43e5-86b4-470524cd5434 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fb787f6e0a8414787a501e06bdfba6f', 'request_id': 'req-fea60d90-3667-43e5-86b4-470524cd5434'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:13.510652 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-fea60d90-3667-43e5-86b4-470524cd5434 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fb787f6e0a8414787a501e06bdfba6f', 'request_id': 'req-fea60d90-3667-43e5-86b4-470524cd5434'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:13.522561 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-fea60d90-3667-43e5-86b4-470524cd5434 admin admin] The token cache is being invalidate because project 7fb787f6e0a8414787a501e06bdfba6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:14:13.522893 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-fea60d90-3667-43e5-86b4-470524cd5434 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:13.523553 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-fea60d90-3667-43e5-86b4-470524cd5434 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:13.525280 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 960/1921] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:14:13 2022] DELETE /identity/v3/projects/7fb787f6e0a8414787a501e06bdfba6f => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:13.529084 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb40bf-3f9a-4e55-b727-b58cf7060944 service neutron] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:13.541588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb40bf-3f9a-4e55-b727-b58cf7060944 service neutron] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:13.607889 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2af2d0-0e52-41d2-b7e2-1bfe83224792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:13.609083 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2af2d0-0e52-41d2-b7e2-1bfe83224792 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:13.609665 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2af2d0-0e52-41d2-b7e2-1bfe83224792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:13.610109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2af2d0-0e52-41d2-b7e2-1bfe83224792 admin admin] PATH_INFO: `/v3/users/3e2c2bdaa94a4b64aed0277d694f8cdc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:13.612011 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2af2d0-0e52-41d2-b7e2-1bfe83224792 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e2c2bdaa94a4b64aed0277d694f8cdc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:13.615170 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2af2d0-0e52-41d2-b7e2-1bfe83224792 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:13.670095 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-8e2af2d0-0e52-41d2-b7e2-1bfe83224792 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e2c2bdaa94a4b64aed0277d694f8cdc'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:13.672718 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:13.677793 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-8e2af2d0-0e52-41d2-b7e2-1bfe83224792 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e2c2bdaa94a4b64aed0277d694f8cdc'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:13.678450 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-8e2af2d0-0e52-41d2-b7e2-1bfe83224792 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e2c2bdaa94a4b64aed0277d694f8cdc'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:13.679303 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-8e2af2d0-0e52-41d2-b7e2-1bfe83224792 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e2c2bdaa94a4b64aed0277d694f8cdc'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:13.688178 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-8e2af2d0-0e52-41d2-b7e2-1bfe83224792 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e2c2bdaa94a4b64aed0277d694f8cdc'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:13.706359 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 962/1922] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:14:13 2022] DELETE /identity/v3/users/3e2c2bdaa94a4b64aed0277d694f8cdc => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:13.976057 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea60d90-3667-43e5-86b4-470524cd5434 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:13.977610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-fea60d90-3667-43e5-86b4-470524cd5434 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:13.986435 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:13.992377 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea60d90-3667-43e5-86b4-470524cd5434 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:14.054224 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c1fdfa-3eca-4084-94d3-0fd356968d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:14.054899 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1fdfa-3eca-4084-94d3-0fd356968d8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:14.055183 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1fdfa-3eca-4084-94d3-0fd356968d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:14.055462 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1fdfa-3eca-4084-94d3-0fd356968d8f admin admin] PATH_INFO: `/v3/projects/094ca5d83ac04bb38d799646353515f9` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:14.056578 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c1fdfa-3eca-4084-94d3-0fd356968d8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=094ca5d83ac04bb38d799646353515f9)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:14.058537 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c1fdfa-3eca-4084-94d3-0fd356968d8f admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:14.121844 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c2c1fdfa-3eca-4084-94d3-0fd356968d8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '094ca5d83ac04bb38d799646353515f9', 'request_id': 'req-c2c1fdfa-3eca-4084-94d3-0fd356968d8f'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:14.133361 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c2c1fdfa-3eca-4084-94d3-0fd356968d8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '094ca5d83ac04bb38d799646353515f9', 'request_id': 'req-c2c1fdfa-3eca-4084-94d3-0fd356968d8f'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:14.144239 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c2c1fdfa-3eca-4084-94d3-0fd356968d8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '094ca5d83ac04bb38d799646353515f9', 'request_id': 'req-c2c1fdfa-3eca-4084-94d3-0fd356968d8f'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:14.157310 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c2c1fdfa-3eca-4084-94d3-0fd356968d8f admin admin] The token cache is being invalidate because project 094ca5d83ac04bb38d799646353515f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:14:14.158089 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c2c1fdfa-3eca-4084-94d3-0fd356968d8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:14.159035 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c2c1fdfa-3eca-4084-94d3-0fd356968d8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:14.160687 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 961/1923] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:14:13 2022] DELETE /identity/v3/projects/094ca5d83ac04bb38d799646353515f9 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:14.168293 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7cbd5e-c9f0-4a6f-a9f5-8f962cae4f00 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:14.168582 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7cbd5e-c9f0-4a6f-a9f5-8f962cae4f00 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:14.168863 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7cbd5e-c9f0-4a6f-a9f5-8f962cae4f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:14.196724 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-9f7cbd5e-c9f0-4a6f-a9f5-8f962cae4f00 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:14.198111 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-9f7cbd5e-c9f0-4a6f-a9f5-8f962cae4f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:14.286211 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 963/1924] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:14.294561 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70541682-42c7-4f10-842f-e42d94adc380 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:14.294815 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70541682-42c7-4f10-842f-e42d94adc380 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:14.295114 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70541682-42c7-4f10-842f-e42d94adc380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:14.328152 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-70541682-42c7-4f10-842f-e42d94adc380 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:14.329383 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-70541682-42c7-4f10-842f-e42d94adc380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:14.339551 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 962/1925] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:14.346758 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7cbd5e-c9f0-4a6f-a9f5-8f962cae4f00 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:14.363816 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7cbd5e-c9f0-4a6f-a9f5-8f962cae4f00 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:14.372501 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa23320-5560-4c47-a2fb-713e2815add1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:14.373135 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa23320-5560-4c47-a2fb-713e2815add1 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:14.373592 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa23320-5560-4c47-a2fb-713e2815add1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:14.373875 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa23320-5560-4c47-a2fb-713e2815add1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:14.375894 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa23320-5560-4c47-a2fb-713e2815add1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:14:14.376267 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa23320-5560-4c47-a2fb-713e2815add1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:14.379410 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa23320-5560-4c47-a2fb-713e2815add1 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:14.401524 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 964/1926] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:14:14.402981 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70541682-42c7-4f10-842f-e42d94adc380 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:14.414526 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70541682-42c7-4f10-842f-e42d94adc380 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:14.419436 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab2210-770d-4632-8622-eef5534363e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:14.419921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab2210-770d-4632-8622-eef5534363e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:14.420203 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab2210-770d-4632-8622-eef5534363e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:14.420475 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab2210-770d-4632-8622-eef5534363e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:14.421113 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ab2210-770d-4632-8622-eef5534363e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:14.423614 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ab2210-770d-4632-8622-eef5534363e8 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:14.474870 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 963/1927] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:14:14 2022] POST /identity/v3/projects => generated 381 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:14:14.477281 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa23320-5560-4c47-a2fb-713e2815add1 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:14.488087 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa23320-5560-4c47-a2fb-713e2815add1 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:14.545982 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832e2981-2fb3-4383-b68f-c0fccb51ae77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:14.547224 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e2981-2fb3-4383-b68f-c0fccb51ae77 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:14.547442 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e2981-2fb3-4383-b68f-c0fccb51ae77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:14.547710 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e2981-2fb3-4383-b68f-c0fccb51ae77 admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:14.548328 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832e2981-2fb3-4383-b68f-c0fccb51ae77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:14.550227 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832e2981-2fb3-4383-b68f-c0fccb51ae77 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:14.572969 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 965/1928] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:14:14 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:14:14.577060 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab2210-770d-4632-8622-eef5534363e8 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:14.587913 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab2210-770d-4632-8622-eef5534363e8 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:14.592825 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e3d40d-603d-4834-8807-836b6781aa66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:14.593958 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e3d40d-603d-4834-8807-836b6781aa66 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:14.594231 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e3d40d-603d-4834-8807-836b6781aa66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:14.594499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e3d40d-603d-4834-8807-836b6781aa66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:14.595170 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e3d40d-603d-4834-8807-836b6781aa66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:14.596735 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e3d40d-603d-4834-8807-836b6781aa66 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:14.606447 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 964/1929] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:14:14.610870 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832e2981-2fb3-4383-b68f-c0fccb51ae77 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:14.620563 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832e2981-2fb3-4383-b68f-c0fccb51ae77 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:14.625119 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c320bd4-8b6e-446c-bd80-db3166f0f563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:14.625820 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c320bd4-8b6e-446c-bd80-db3166f0f563 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:14.626129 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c320bd4-8b6e-446c-bd80-db3166f0f563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:14.626429 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c320bd4-8b6e-446c-bd80-db3166f0f563 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:14.627093 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c320bd4-8b6e-446c-bd80-db3166f0f563 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:14.628654 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c320bd4-8b6e-446c-bd80-db3166f0f563 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:14.642793 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 966/1930] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:14:14.643997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e3d40d-603d-4834-8807-836b6781aa66 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:14.655190 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e3d40d-603d-4834-8807-836b6781aa66 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:14.660212 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe59b9bc-16a2-401b-ab39-15c4b02fed93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:14.660855 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59b9bc-16a2-401b-ab39-15c4b02fed93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:14.661127 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59b9bc-16a2-401b-ab39-15c4b02fed93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:14.661394 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59b9bc-16a2-401b-ab39-15c4b02fed93 admin admin] PATH_INFO: `/v3/projects/045ea67b1974438bae4ac6df5c31a4fd/users/aeb9dea816a444e2ac2a69ff83b3e480/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:14.676299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe59b9bc-16a2-401b-ab39-15c4b02fed93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=045ea67b1974438bae4ac6df5c31a4fd, user_id=aeb9dea816a444e2ac2a69ff83b3e480, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:14.678925 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe59b9bc-16a2-401b-ab39-15c4b02fed93 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:14.691420 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 965/1931] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:14:14 2022] PUT /identity/v3/projects/045ea67b1974438bae4ac6df5c31a4fd/users/aeb9dea816a444e2ac2a69ff83b3e480/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:14:14.702445 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d15f6-b6d0-48a9-b2a1-c65988fa9867 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:14.702760 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d15f6-b6d0-48a9-b2a1-c65988fa9867 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:14.703084 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d15f6-b6d0-48a9-b2a1-c65988fa9867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:14.731170 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-066d15f6-b6d0-48a9-b2a1-c65988fa9867 None None] MFA Rules not processed for user `aeb9dea816a444e2ac2a69ff83b3e480`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:14.732362 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-066d15f6-b6d0-48a9-b2a1-c65988fa9867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:14.752374 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:14.797339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 967/1932] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:14.801902 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe59b9bc-16a2-401b-ab39-15c4b02fed93 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:14.832243 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:14.865405 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe59b9bc-16a2-401b-ab39-15c4b02fed93 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:14.871681 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666df392-3e91-4d25-9f01-91b8d2219fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:14.872232 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666df392-3e91-4d25-9f01-91b8d2219fa5 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:14.872511 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666df392-3e91-4d25-9f01-91b8d2219fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:14.872783 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666df392-3e91-4d25-9f01-91b8d2219fa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:14.873422 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666df392-3e91-4d25-9f01-91b8d2219fa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:14.875372 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666df392-3e91-4d25-9f01-91b8d2219fa5 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:14.905053 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 966/1933] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:14:14 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:14:14.909770 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d15f6-b6d0-48a9-b2a1-c65988fa9867 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:14.920259 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d15f6-b6d0-48a9-b2a1-c65988fa9867 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:14.975472 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02435dc-ae41-4b49-9d96-b0a4f7e17b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:14.976088 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02435dc-ae41-4b49-9d96-b0a4f7e17b40 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:14.976366 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02435dc-ae41-4b49-9d96-b0a4f7e17b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:14.976637 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02435dc-ae41-4b49-9d96-b0a4f7e17b40 admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:14.977269 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02435dc-ae41-4b49-9d96-b0a4f7e17b40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:14.979205 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02435dc-ae41-4b49-9d96-b0a4f7e17b40 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:15.004916 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 968/1934] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:14:14 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:14:15.006127 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666df392-3e91-4d25-9f01-91b8d2219fa5 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:15.016663 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666df392-3e91-4d25-9f01-91b8d2219fa5 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:15.021571 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37838b0-808e-40eb-9076-b360b056dcb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:15.022219 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37838b0-808e-40eb-9076-b360b056dcb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:15.022498 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37838b0-808e-40eb-9076-b360b056dcb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:15.022812 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37838b0-808e-40eb-9076-b360b056dcb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:15.023614 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37838b0-808e-40eb-9076-b360b056dcb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:15.025344 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37838b0-808e-40eb-9076-b360b056dcb8 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:15.037266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 967/1935] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:14:15.041339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02435dc-ae41-4b49-9d96-b0a4f7e17b40 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:15.051309 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02435dc-ae41-4b49-9d96-b0a4f7e17b40 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:15.055893 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9116aa-3a44-4b43-9f66-6a2a414f0379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:15.056560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9116aa-3a44-4b43-9f66-6a2a414f0379 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:15.056838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9116aa-3a44-4b43-9f66-6a2a414f0379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:15.057109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9116aa-3a44-4b43-9f66-6a2a414f0379 admin admin] PATH_INFO: `/v3/projects/cb7fde5be83d43e493c5ab426ab40ea5/users/e9dca6236d4e49c28c99be34ec8a49eb/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:15.073492 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9116aa-3a44-4b43-9f66-6a2a414f0379 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb7fde5be83d43e493c5ab426ab40ea5, user_id=e9dca6236d4e49c28c99be34ec8a49eb, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:15.076203 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9116aa-3a44-4b43-9f66-6a2a414f0379 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:15.089760 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 969/1936] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:14:15 2022] PUT /identity/v3/projects/cb7fde5be83d43e493c5ab426ab40ea5/users/e9dca6236d4e49c28c99be34ec8a49eb/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:15.094215 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37838b0-808e-40eb-9076-b360b056dcb8 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:15.104115 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37838b0-808e-40eb-9076-b360b056dcb8 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:15.166013 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e07d2f-6ea9-4266-ba72-0d0d5c563879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:15.166623 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e07d2f-6ea9-4266-ba72-0d0d5c563879 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:15.166926 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e07d2f-6ea9-4266-ba72-0d0d5c563879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:15.167261 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e07d2f-6ea9-4266-ba72-0d0d5c563879 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:15.167836 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e07d2f-6ea9-4266-ba72-0d0d5c563879 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:15.169406 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e07d2f-6ea9-4266-ba72-0d0d5c563879 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:15.180242 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 968/1937] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:14:15.184887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9116aa-3a44-4b43-9f66-6a2a414f0379 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:15.195937 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9116aa-3a44-4b43-9f66-6a2a414f0379 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:15.200429 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d808c643-1af8-430e-bac1-10357bcf5845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:15.201070 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808c643-1af8-430e-bac1-10357bcf5845 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:15.201347 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808c643-1af8-430e-bac1-10357bcf5845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:15.201616 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808c643-1af8-430e-bac1-10357bcf5845 admin admin] PATH_INFO: `/v3/projects/cb7fde5be83d43e493c5ab426ab40ea5/users/e9dca6236d4e49c28c99be34ec8a49eb/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:15.204167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d808c643-1af8-430e-bac1-10357bcf5845 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb7fde5be83d43e493c5ab426ab40ea5, user_id=e9dca6236d4e49c28c99be34ec8a49eb, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:15.206764 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d808c643-1af8-430e-bac1-10357bcf5845 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:15.216747 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 970/1938] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:14:15 2022] PUT /identity/v3/projects/cb7fde5be83d43e493c5ab426ab40ea5/users/e9dca6236d4e49c28c99be34ec8a49eb/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:15.226543 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8a23c-e883-478a-b5be-a8cc30789094 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:15.226827 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8a23c-e883-478a-b5be-a8cc30789094 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:15.227128 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8a23c-e883-478a-b5be-a8cc30789094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:15.255170 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-00c8a23c-e883-478a-b5be-a8cc30789094 None None] MFA Rules not processed for user `e9dca6236d4e49c28c99be34ec8a49eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:15.256547 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-00c8a23c-e883-478a-b5be-a8cc30789094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:15.328995 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 969/1939] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:15 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:15.339913 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d808c643-1af8-430e-bac1-10357bcf5845 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:15.341714 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-d808c643-1af8-430e-bac1-10357bcf5845 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:15.401997 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d808c643-1af8-430e-bac1-10357bcf5845 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:15.405498 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d06a96-7c22-433f-be84-1ceb8bfe824f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:15.406134 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d06a96-7c22-433f-be84-1ceb8bfe824f service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:15.406407 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d06a96-7c22-433f-be84-1ceb8bfe824f service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:15.406673 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d06a96-7c22-433f-be84-1ceb8bfe824f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:15.415946 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8a23c-e883-478a-b5be-a8cc30789094 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:15.418549 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-00c8a23c-e883-478a-b5be-a8cc30789094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:15.420205 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d06a96-7c22-433f-be84-1ceb8bfe824f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:15.421658 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d06a96-7c22-433f-be84-1ceb8bfe824f service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:15.432868 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 971/1940] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:14:15 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:14:15.450679 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:15.541798 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8a23c-e883-478a-b5be-a8cc30789094 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:15.547628 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf69ba7-2ebb-4e52-822b-8c4e87b85018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:15.548344 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf69ba7-2ebb-4e52-822b-8c4e87b85018 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:15.548709 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf69ba7-2ebb-4e52-822b-8c4e87b85018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:15.549007 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf69ba7-2ebb-4e52-822b-8c4e87b85018 admin admin] PATH_INFO: `/v3/users/a2357ca63bff41509e18ed10d9b2214c` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:15.550150 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf69ba7-2ebb-4e52-822b-8c4e87b85018 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2357ca63bff41509e18ed10d9b2214c)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:15.552092 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf69ba7-2ebb-4e52-822b-8c4e87b85018 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:15.614186 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-8bf69ba7-2ebb-4e52-822b-8c4e87b85018 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2357ca63bff41509e18ed10d9b2214c'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:15.628149 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-8bf69ba7-2ebb-4e52-822b-8c4e87b85018 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2357ca63bff41509e18ed10d9b2214c'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:15.629223 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-8bf69ba7-2ebb-4e52-822b-8c4e87b85018 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2357ca63bff41509e18ed10d9b2214c'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:15.630278 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-8bf69ba7-2ebb-4e52-822b-8c4e87b85018 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2357ca63bff41509e18ed10d9b2214c'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:15.640675 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-8bf69ba7-2ebb-4e52-822b-8c4e87b85018 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2357ca63bff41509e18ed10d9b2214c'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:15.685320 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 970/1941] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:14:15 2022] DELETE /identity/v3/users/a2357ca63bff41509e18ed10d9b2214c => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:15.687084 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cbc72-9d91-4913-b135-a2fa06d35737 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:15.688016 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cbc72-9d91-4913-b135-a2fa06d35737 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:15.688234 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cbc72-9d91-4913-b135-a2fa06d35737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:15.708639 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-ce6cbc72-9d91-4913-b135-a2fa06d35737 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:15.710418 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-ce6cbc72-9d91-4913-b135-a2fa06d35737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:15.726796 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:15.786504 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 972/1942] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:15.801296 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf69ba7-2ebb-4e52-822b-8c4e87b85018 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:15.808526 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-8bf69ba7-2ebb-4e52-822b-8c4e87b85018 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:15.898700 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6cbc72-9d91-4913-b135-a2fa06d35737 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:15.901321 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-ce6cbc72-9d91-4913-b135-a2fa06d35737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:15.901752 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf69ba7-2ebb-4e52-822b-8c4e87b85018 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:15.905433 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904f910-6858-4c6e-9055-c255e1f961a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:15.906582 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904f910-6858-4c6e-9055-c255e1f961a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:15.906895 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904f910-6858-4c6e-9055-c255e1f961a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:15.907262 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904f910-6858-4c6e-9055-c255e1f961a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:15.915439 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6cbc72-9d91-4913-b135-a2fa06d35737 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:15.918922 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904f910-6858-4c6e-9055-c255e1f961a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:15.920357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939da3a6-5490-4ee6-8850-47ddb3d5ba69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:15.920478 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904f910-6858-4c6e-9055-c255e1f961a2 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:15.921218 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939da3a6-5490-4ee6-8850-47ddb3d5ba69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:15.922078 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939da3a6-5490-4ee6-8850-47ddb3d5ba69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:15.922351 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939da3a6-5490-4ee6-8850-47ddb3d5ba69 admin admin] PATH_INFO: `/v3/users/aeb9dea816a444e2ac2a69ff83b3e480` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:15.923536 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939da3a6-5490-4ee6-8850-47ddb3d5ba69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aeb9dea816a444e2ac2a69ff83b3e480)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:15.925421 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939da3a6-5490-4ee6-8850-47ddb3d5ba69 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:15.931432 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 971/1943] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:14:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:14:15.982086 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-939da3a6-5490-4ee6-8850-47ddb3d5ba69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aeb9dea816a444e2ac2a69ff83b3e480'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:15.997163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-939da3a6-5490-4ee6-8850-47ddb3d5ba69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aeb9dea816a444e2ac2a69ff83b3e480'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:15.998963 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-939da3a6-5490-4ee6-8850-47ddb3d5ba69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aeb9dea816a444e2ac2a69ff83b3e480'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:15.999633 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-939da3a6-5490-4ee6-8850-47ddb3d5ba69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aeb9dea816a444e2ac2a69ff83b3e480'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:16.010500 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-939da3a6-5490-4ee6-8850-47ddb3d5ba69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aeb9dea816a444e2ac2a69ff83b3e480'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:16.040086 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 973/1944] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:14:15 2022] DELETE /identity/v3/users/aeb9dea816a444e2ac2a69ff83b3e480 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:14:16.043284 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c7920-f70b-4437-b1df-3fb235882841 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:16.043725 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c7920-f70b-4437-b1df-3fb235882841 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:16.043957 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c7920-f70b-4437-b1df-3fb235882841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:16.077405 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-795c7920-f70b-4437-b1df-3fb235882841 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:16.078712 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-795c7920-f70b-4437-b1df-3fb235882841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:16.137630 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:16.175566 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 972/1945] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:16.187820 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939da3a6-5490-4ee6-8850-47ddb3d5ba69 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:16.189502 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-939da3a6-5490-4ee6-8850-47ddb3d5ba69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:16.253923 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939da3a6-5490-4ee6-8850-47ddb3d5ba69 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:16.257842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca480185-9c76-4b93-a45e-1518aa22604e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:16.258480 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca480185-9c76-4b93-a45e-1518aa22604e service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:16.258761 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca480185-9c76-4b93-a45e-1518aa22604e service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:16.259066 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca480185-9c76-4b93-a45e-1518aa22604e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:16.274186 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca480185-9c76-4b93-a45e-1518aa22604e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:16.274936 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca480185-9c76-4b93-a45e-1518aa22604e service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:16.289705 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 974/1946] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:14:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:14:16.311832 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c7920-f70b-4437-b1df-3fb235882841 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:16.313626 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-795c7920-f70b-4437-b1df-3fb235882841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:16.326697 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c7920-f70b-4437-b1df-3fb235882841 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:16.340845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac9b7d6-8a06-442e-bca9-916ea7ee289d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:16.341523 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9b7d6-8a06-442e-bca9-916ea7ee289d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:16.342219 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9b7d6-8a06-442e-bca9-916ea7ee289d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:16.342219 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9b7d6-8a06-442e-bca9-916ea7ee289d admin admin] PATH_INFO: `/v3/projects/0f9f744449744d959abd33704b09fac8` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:16.345444 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac9b7d6-8a06-442e-bca9-916ea7ee289d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f9f744449744d959abd33704b09fac8)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:16.348451 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac9b7d6-8a06-442e-bca9-916ea7ee289d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:16.428665 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-dac9b7d6-8a06-442e-bca9-916ea7ee289d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f9f744449744d959abd33704b09fac8', 'request_id': 'req-dac9b7d6-8a06-442e-bca9-916ea7ee289d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:16.438445 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-dac9b7d6-8a06-442e-bca9-916ea7ee289d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f9f744449744d959abd33704b09fac8', 'request_id': 'req-dac9b7d6-8a06-442e-bca9-916ea7ee289d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:16.445063 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-dac9b7d6-8a06-442e-bca9-916ea7ee289d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f9f744449744d959abd33704b09fac8', 'request_id': 'req-dac9b7d6-8a06-442e-bca9-916ea7ee289d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:16.469253 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-dac9b7d6-8a06-442e-bca9-916ea7ee289d admin admin] The token cache is being invalidate because project 0f9f744449744d959abd33704b09fac8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:14:16.471251 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-dac9b7d6-8a06-442e-bca9-916ea7ee289d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:16.473009 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-dac9b7d6-8a06-442e-bca9-916ea7ee289d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:16.476061 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 973/1947] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:14:16 2022] DELETE /identity/v3/projects/0f9f744449744d959abd33704b09fac8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:14:16.480592 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca480185-9c76-4b93-a45e-1518aa22604e service neutron] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:16.492480 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca480185-9c76-4b93-a45e-1518aa22604e service neutron] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:16.527039 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:16.565394 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6e73f-c1b0-4473-99fa-f4a65eed626b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:16.565394 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6e73f-c1b0-4473-99fa-f4a65eed626b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:16.566031 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6e73f-c1b0-4473-99fa-f4a65eed626b admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:16.566707 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6e73f-c1b0-4473-99fa-f4a65eed626b admin admin] PATH_INFO: `/v3/users/5ef624a43d9b4d87970e896212202f5c` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:16.568282 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be6e73f-c1b0-4473-99fa-f4a65eed626b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ef624a43d9b4d87970e896212202f5c)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:16.570903 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be6e73f-c1b0-4473-99fa-f4a65eed626b admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:16.608731 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac9b7d6-8a06-442e-bca9-916ea7ee289d admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:16.611048 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-dac9b7d6-8a06-442e-bca9-916ea7ee289d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:16.633823 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac9b7d6-8a06-442e-bca9-916ea7ee289d admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:16.639911 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e18e24-794a-48a9-81de-6f4eab28c4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:16.640660 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e18e24-794a-48a9-81de-6f4eab28c4b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:16.641003 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e18e24-794a-48a9-81de-6f4eab28c4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:16.641338 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e18e24-794a-48a9-81de-6f4eab28c4b8 admin admin] PATH_INFO: `/v3/projects/045ea67b1974438bae4ac6df5c31a4fd` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:16.642551 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e18e24-794a-48a9-81de-6f4eab28c4b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=045ea67b1974438bae4ac6df5c31a4fd)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:16.645899 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e18e24-794a-48a9-81de-6f4eab28c4b8 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:16.656946 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-3be6e73f-c1b0-4473-99fa-f4a65eed626b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ef624a43d9b4d87970e896212202f5c'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:16.673917 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-3be6e73f-c1b0-4473-99fa-f4a65eed626b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ef624a43d9b4d87970e896212202f5c'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:16.677098 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-3be6e73f-c1b0-4473-99fa-f4a65eed626b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ef624a43d9b4d87970e896212202f5c'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:16.677957 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-3be6e73f-c1b0-4473-99fa-f4a65eed626b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ef624a43d9b4d87970e896212202f5c'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:16.688629 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-3be6e73f-c1b0-4473-99fa-f4a65eed626b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ef624a43d9b4d87970e896212202f5c'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:16.715618 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 975/1948] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:14:16 2022] DELETE /identity/v3/users/5ef624a43d9b4d87970e896212202f5c => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:16.727234 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-14e18e24-794a-48a9-81de-6f4eab28c4b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '045ea67b1974438bae4ac6df5c31a4fd', 'request_id': 'req-14e18e24-794a-48a9-81de-6f4eab28c4b8'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:16.738351 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-14e18e24-794a-48a9-81de-6f4eab28c4b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '045ea67b1974438bae4ac6df5c31a4fd', 'request_id': 'req-14e18e24-794a-48a9-81de-6f4eab28c4b8'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:16.745173 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-14e18e24-794a-48a9-81de-6f4eab28c4b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '045ea67b1974438bae4ac6df5c31a4fd', 'request_id': 'req-14e18e24-794a-48a9-81de-6f4eab28c4b8'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:16.762222 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-14e18e24-794a-48a9-81de-6f4eab28c4b8 admin admin] The token cache is being invalidate because project 045ea67b1974438bae4ac6df5c31a4fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:14:16.762708 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-14e18e24-794a-48a9-81de-6f4eab28c4b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:16.763661 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-14e18e24-794a-48a9-81de-6f4eab28c4b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:16.766905 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 974/1949] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:14:16 2022] DELETE /identity/v3/projects/045ea67b1974438bae4ac6df5c31a4fd => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:14:16.769884 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6e73f-c1b0-4473-99fa-f4a65eed626b admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:16.774464 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-3be6e73f-c1b0-4473-99fa-f4a65eed626b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:16.860129 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6e73f-c1b0-4473-99fa-f4a65eed626b admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:16.865788 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139fc7c6-6a04-473a-ade1-70eecf6fe723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:16.866575 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139fc7c6-6a04-473a-ade1-70eecf6fe723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:16.866951 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139fc7c6-6a04-473a-ade1-70eecf6fe723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:16.867308 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139fc7c6-6a04-473a-ade1-70eecf6fe723 admin admin] PATH_INFO: `/v3/users/e9dca6236d4e49c28c99be34ec8a49eb` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:16.868625 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139fc7c6-6a04-473a-ade1-70eecf6fe723 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9dca6236d4e49c28c99be34ec8a49eb)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:16.870534 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139fc7c6-6a04-473a-ade1-70eecf6fe723 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:16.933573 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-139fc7c6-6a04-473a-ade1-70eecf6fe723 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9dca6236d4e49c28c99be34ec8a49eb'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:16.946501 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-139fc7c6-6a04-473a-ade1-70eecf6fe723 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9dca6236d4e49c28c99be34ec8a49eb'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:16.947295 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-139fc7c6-6a04-473a-ade1-70eecf6fe723 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9dca6236d4e49c28c99be34ec8a49eb'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:16.947946 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-139fc7c6-6a04-473a-ade1-70eecf6fe723 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9dca6236d4e49c28c99be34ec8a49eb'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:16.949813 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:16.959284 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-139fc7c6-6a04-473a-ade1-70eecf6fe723 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9dca6236d4e49c28c99be34ec8a49eb'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:16.987786 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 976/1950] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:14:16 2022] DELETE /identity/v3/users/e9dca6236d4e49c28c99be34ec8a49eb => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:17.086752 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e18e24-794a-48a9-81de-6f4eab28c4b8 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:17.088321 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-14e18e24-794a-48a9-81de-6f4eab28c4b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:17.191116 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:17.212548 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e18e24-794a-48a9-81de-6f4eab28c4b8 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:17.302426 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e73af5-4c9b-4014-bd2c-8c9d8471bd09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:17.303340 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e73af5-4c9b-4014-bd2c-8c9d8471bd09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:17.303623 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e73af5-4c9b-4014-bd2c-8c9d8471bd09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:17.303900 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e73af5-4c9b-4014-bd2c-8c9d8471bd09 admin admin] PATH_INFO: `/v3/projects/1a7398341fec445ab429c299872c1ab5` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:17.305069 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e73af5-4c9b-4014-bd2c-8c9d8471bd09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a7398341fec445ab429c299872c1ab5)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:17.306992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e73af5-4c9b-4014-bd2c-8c9d8471bd09 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:17.376127 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139fc7c6-6a04-473a-ade1-70eecf6fe723 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:17.381046 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-139fc7c6-6a04-473a-ade1-70eecf6fe723 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:17.383836 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-47e73af5-4c9b-4014-bd2c-8c9d8471bd09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a7398341fec445ab429c299872c1ab5', 'request_id': 'req-47e73af5-4c9b-4014-bd2c-8c9d8471bd09'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:17.393601 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-47e73af5-4c9b-4014-bd2c-8c9d8471bd09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a7398341fec445ab429c299872c1ab5', 'request_id': 'req-47e73af5-4c9b-4014-bd2c-8c9d8471bd09'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:17.400208 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-47e73af5-4c9b-4014-bd2c-8c9d8471bd09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a7398341fec445ab429c299872c1ab5', 'request_id': 'req-47e73af5-4c9b-4014-bd2c-8c9d8471bd09'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:17.413629 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139fc7c6-6a04-473a-ade1-70eecf6fe723 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:17.417955 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-47e73af5-4c9b-4014-bd2c-8c9d8471bd09 admin admin] The token cache is being invalidate because project 1a7398341fec445ab429c299872c1ab5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:14:17.418305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-47e73af5-4c9b-4014-bd2c-8c9d8471bd09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:17.419017 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-47e73af5-4c9b-4014-bd2c-8c9d8471bd09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:17.421498 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 975/1951] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:14:17 2022] DELETE /identity/v3/projects/1a7398341fec445ab429c299872c1ab5 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:17.426143 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcae4bf-c2e7-4285-8f9b-1a3349ea6231 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:17.426431 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcae4bf-c2e7-4285-8f9b-1a3349ea6231 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:17.426707 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcae4bf-c2e7-4285-8f9b-1a3349ea6231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:17.449497 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-3bcae4bf-c2e7-4285-8f9b-1a3349ea6231 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:17.450768 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-3bcae4bf-c2e7-4285-8f9b-1a3349ea6231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:17.480321 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc6b3c-d15e-4a78-9df7-9d1b88d12e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:17.481009 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc6b3c-d15e-4a78-9df7-9d1b88d12e37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:17.481290 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc6b3c-d15e-4a78-9df7-9d1b88d12e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:17.481562 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc6b3c-d15e-4a78-9df7-9d1b88d12e37 admin admin] PATH_INFO: `/v3/projects/cb7fde5be83d43e493c5ab426ab40ea5` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:17.482877 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bc6b3c-d15e-4a78-9df7-9d1b88d12e37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb7fde5be83d43e493c5ab426ab40ea5)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:17.484962 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bc6b3c-d15e-4a78-9df7-9d1b88d12e37 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:17.529749 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 976/1952] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:17.538938 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4a203-255d-4ff1-8f49-94d0adc17d81 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:17.539481 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4a203-255d-4ff1-8f49-94d0adc17d81 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:17.540001 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4a203-255d-4ff1-8f49-94d0adc17d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:17.547594 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-b1bc6b3c-d15e-4a78-9df7-9d1b88d12e37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb7fde5be83d43e493c5ab426ab40ea5', 'request_id': 'req-b1bc6b3c-d15e-4a78-9df7-9d1b88d12e37'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:17.555753 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-b1bc6b3c-d15e-4a78-9df7-9d1b88d12e37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb7fde5be83d43e493c5ab426ab40ea5', 'request_id': 'req-b1bc6b3c-d15e-4a78-9df7-9d1b88d12e37'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:17.571484 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-b1bc6b3c-d15e-4a78-9df7-9d1b88d12e37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb7fde5be83d43e493c5ab426ab40ea5', 'request_id': 'req-b1bc6b3c-d15e-4a78-9df7-9d1b88d12e37'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:17.574444 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-65a4a203-255d-4ff1-8f49-94d0adc17d81 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:17.575828 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-65a4a203-255d-4ff1-8f49-94d0adc17d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:17.585973 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-b1bc6b3c-d15e-4a78-9df7-9d1b88d12e37 admin admin] The token cache is being invalidate because project cb7fde5be83d43e493c5ab426ab40ea5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:14:17.586261 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-b1bc6b3c-d15e-4a78-9df7-9d1b88d12e37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:17.587265 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-b1bc6b3c-d15e-4a78-9df7-9d1b88d12e37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:17.589903 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 977/1953] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:14:17 2022] DELETE /identity/v3/projects/cb7fde5be83d43e493c5ab426ab40ea5 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:17.595830 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd0e15-8f61-4374-abb3-b76936323355 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:17.595830 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd0e15-8f61-4374-abb3-b76936323355 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:17.596069 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd0e15-8f61-4374-abb3-b76936323355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:17.617650 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:17.630608 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-38bd0e15-8f61-4374-abb3-b76936323355 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:17.631904 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-38bd0e15-8f61-4374-abb3-b76936323355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:17.659746 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 977/1954] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:17.664112 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4a203-255d-4ff1-8f49-94d0adc17d81 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:17.673721 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4a203-255d-4ff1-8f49-94d0adc17d81 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:17.679260 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a4c98-7743-453c-b67f-f1a5f028a4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:17.679869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a4c98-7743-453c-b67f-f1a5f028a4b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:17.680142 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a4c98-7743-453c-b67f-f1a5f028a4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:17.680408 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a4c98-7743-453c-b67f-f1a5f028a4b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:17.681053 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a4c98-7743-453c-b67f-f1a5f028a4b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:14:17.681348 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a4c98-7743-453c-b67f-f1a5f028a4b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:17.682938 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a4c98-7743-453c-b67f-f1a5f028a4b8 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:17.695778 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 978/1955] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:14:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:14:17.700023 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a4c98-7743-453c-b67f-f1a5f028a4b8 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:17.711311 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a4c98-7743-453c-b67f-f1a5f028a4b8 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:17.712425 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 978/1956] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:14:17.720967 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9594ec-91a8-4cc2-bece-cea9c4e511f5 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:17.721335 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9594ec-91a8-4cc2-bece-cea9c4e511f5 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:17.721731 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9594ec-91a8-4cc2-bece-cea9c4e511f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:17.723253 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce476751-5d20-4db9-bdef-b39814a395e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:17.723882 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce476751-5d20-4db9-bdef-b39814a395e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:17.724225 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce476751-5d20-4db9-bdef-b39814a395e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:17.724498 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce476751-5d20-4db9-bdef-b39814a395e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:17.725127 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce476751-5d20-4db9-bdef-b39814a395e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:17.727263 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce476751-5d20-4db9-bdef-b39814a395e0 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:17.729700 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:17.745520 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-ce9594ec-91a8-4cc2-bece-cea9c4e511f5 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:17.746742 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-ce9594ec-91a8-4cc2-bece-cea9c4e511f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:17.757298 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 979/1957] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:14:17.761696 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9594ec-91a8-4cc2-bece-cea9c4e511f5 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:17.763572 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 979/1958] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:14:17 2022] POST /identity/v3/projects => generated 383 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:14:17.768340 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce476751-5d20-4db9-bdef-b39814a395e0 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:17.773051 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9594ec-91a8-4cc2-bece-cea9c4e511f5 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:17.851685 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578ce36-d25c-439e-9faa-b1c0288053ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:17.852311 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578ce36-d25c-439e-9faa-b1c0288053ee admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:17.852623 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578ce36-d25c-439e-9faa-b1c0288053ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:17.852896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578ce36-d25c-439e-9faa-b1c0288053ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:17.853603 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a578ce36-d25c-439e-9faa-b1c0288053ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:14:17.853905 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a578ce36-d25c-439e-9faa-b1c0288053ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:17.855578 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a578ce36-d25c-439e-9faa-b1c0288053ee admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:17.861424 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce476751-5d20-4db9-bdef-b39814a395e0 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:17.871368 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 980/1959] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:14:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:14:17.872801 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578ce36-d25c-439e-9faa-b1c0288053ee admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:17.872980 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee0452-37a2-4f18-9ace-c53c19465ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:17.874287 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee0452-37a2-4f18-9ace-c53c19465ff1 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:17.874287 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee0452-37a2-4f18-9ace-c53c19465ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:17.874287 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee0452-37a2-4f18-9ace-c53c19465ff1 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:17.875508 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee0452-37a2-4f18-9ace-c53c19465ff1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:17.877439 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee0452-37a2-4f18-9ace-c53c19465ff1 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:17.885075 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578ce36-d25c-439e-9faa-b1c0288053ee admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:17.891633 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45fbd4-e0d3-418b-865c-454d9adde518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:17.892284 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45fbd4-e0d3-418b-865c-454d9adde518 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:17.892546 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45fbd4-e0d3-418b-865c-454d9adde518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:17.892859 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45fbd4-e0d3-418b-865c-454d9adde518 admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:17.893574 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e45fbd4-e0d3-418b-865c-454d9adde518 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:17.895782 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e45fbd4-e0d3-418b-865c-454d9adde518 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:17.903349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 980/1960] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:14:17 2022] POST /identity/v3/users => generated 292 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:14:17.904393 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee0452-37a2-4f18-9ace-c53c19465ff1 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:17.915395 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee0452-37a2-4f18-9ace-c53c19465ff1 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:17.920596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eeef9a-10db-4f02-913f-dfab101860cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:17.921207 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eeef9a-10db-4f02-913f-dfab101860cc admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:17.922029 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eeef9a-10db-4f02-913f-dfab101860cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:17.922300 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eeef9a-10db-4f02-913f-dfab101860cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:17.922975 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eeef9a-10db-4f02-913f-dfab101860cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:17.924587 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eeef9a-10db-4f02-913f-dfab101860cc admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:17.929832 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 981/1961] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:14:17 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:14:17.934295 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45fbd4-e0d3-418b-865c-454d9adde518 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:17.939421 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 981/1962] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:14:17.943314 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eeef9a-10db-4f02-913f-dfab101860cc admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:17.955058 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eeef9a-10db-4f02-913f-dfab101860cc admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:18.014529 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45fbd4-e0d3-418b-865c-454d9adde518 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:18.018011 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4465c-741f-4dda-890e-d00b860e7f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:18.018706 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4465c-741f-4dda-890e-d00b860e7f49 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:18.019080 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4465c-741f-4dda-890e-d00b860e7f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:18.019470 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4465c-741f-4dda-890e-d00b860e7f49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:18.020135 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ea093-7ee8-43d2-948b-844c3daf4f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:18.020258 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac4465c-741f-4dda-890e-d00b860e7f49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:18.020872 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ea093-7ee8-43d2-948b-844c3daf4f6c admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:18.021218 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ea093-7ee8-43d2-948b-844c3daf4f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:18.021557 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ea093-7ee8-43d2-948b-844c3daf4f6c admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:18.022095 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac4465c-741f-4dda-890e-d00b860e7f49 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:18.022357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1ea093-7ee8-43d2-948b-844c3daf4f6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:18.025030 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1ea093-7ee8-43d2-948b-844c3daf4f6c admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:18.036003 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 982/1963] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:14:18.037067 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4465c-741f-4dda-890e-d00b860e7f49 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:18.048406 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4465c-741f-4dda-890e-d00b860e7f49 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:18.050886 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 982/1964] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:14:17 2022] POST /identity/v3/users => generated 287 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:14:18.053554 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ea093-7ee8-43d2-948b-844c3daf4f6c admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:18.054372 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabc2195-8f62-45c8-bb0a-dad1ff1158a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:18.055141 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc2195-8f62-45c8-bb0a-dad1ff1158a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:18.055488 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc2195-8f62-45c8-bb0a-dad1ff1158a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:18.055844 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc2195-8f62-45c8-bb0a-dad1ff1158a2 admin admin] PATH_INFO: `/v3/projects/bac4a3d979ea4877a5cc6ce3ca947e3b/users/6abc2245d0dc494da8332ab0fcfca4c6/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:18.064494 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ea093-7ee8-43d2-948b-844c3daf4f6c admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:18.067065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:18.069148 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3763695e-e4d0-4075-8434-75a449c848d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:18.069753 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763695e-e4d0-4075-8434-75a449c848d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:18.070030 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763695e-e4d0-4075-8434-75a449c848d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:18.070420 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763695e-e4d0-4075-8434-75a449c848d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:18.071090 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3763695e-e4d0-4075-8434-75a449c848d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:18.072688 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3763695e-e4d0-4075-8434-75a449c848d3 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:18.076470 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabc2195-8f62-45c8-bb0a-dad1ff1158a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bac4a3d979ea4877a5cc6ce3ca947e3b, user_id=6abc2245d0dc494da8332ab0fcfca4c6, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:18.079459 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabc2195-8f62-45c8-bb0a-dad1ff1158a2 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:18.083929 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 983/1965] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:14:18.088006 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3763695e-e4d0-4075-8434-75a449c848d3 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:18.095506 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 983/1966] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:14:18 2022] PUT /identity/v3/projects/bac4a3d979ea4877a5cc6ce3ca947e3b/users/6abc2245d0dc494da8332ab0fcfca4c6/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:18.098202 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3763695e-e4d0-4075-8434-75a449c848d3 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:18.106627 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be6883-1432-4281-a4c4-6a27558316af None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:18.106952 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be6883-1432-4281-a4c4-6a27558316af None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:18.107223 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be6883-1432-4281-a4c4-6a27558316af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:18.152205 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-a6be6883-1432-4281-a4c4-6a27558316af None None] MFA Rules not processed for user `6abc2245d0dc494da8332ab0fcfca4c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:18.153718 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-a6be6883-1432-4281-a4c4-6a27558316af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:18.154951 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:18.172555 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3142115-1631-4cd8-8c42-111ba892cc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:18.173176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3142115-1631-4cd8-8c42-111ba892cc9f admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:18.173448 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3142115-1631-4cd8-8c42-111ba892cc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:18.173717 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3142115-1631-4cd8-8c42-111ba892cc9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:18.174352 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3142115-1631-4cd8-8c42-111ba892cc9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:18.175955 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3142115-1631-4cd8-8c42-111ba892cc9f admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:18.187443 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 984/1967] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:14:18.191875 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3142115-1631-4cd8-8c42-111ba892cc9f admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:18.202604 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3142115-1631-4cd8-8c42-111ba892cc9f admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:18.208088 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c01483-c552-44c4-ac14-6232d2fdadfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:18.208802 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c01483-c552-44c4-ac14-6232d2fdadfe admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:18.208928 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c01483-c552-44c4-ac14-6232d2fdadfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:18.209117 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c01483-c552-44c4-ac14-6232d2fdadfe admin admin] PATH_INFO: `/v3/projects/9221b61786004e0b8e21b13c18490fed/users/d56b1b3d27c744858093f3ab829b8c81/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:18.225776 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c01483-c552-44c4-ac14-6232d2fdadfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=9221b61786004e0b8e21b13c18490fed, user_id=d56b1b3d27c744858093f3ab829b8c81, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:18.228446 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c01483-c552-44c4-ac14-6232d2fdadfe admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:18.231256 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 984/1968] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:14:18.236094 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be6883-1432-4281-a4c4-6a27558316af None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:18.250361 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be6883-1432-4281-a4c4-6a27558316af None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:18.256564 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d009a83-b6c2-4ce1-a04f-8ff812b1eca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:18.257404 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d009a83-b6c2-4ce1-a04f-8ff812b1eca2 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:18.257779 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d009a83-b6c2-4ce1-a04f-8ff812b1eca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:18.258384 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d009a83-b6c2-4ce1-a04f-8ff812b1eca2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:18.259239 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d009a83-b6c2-4ce1-a04f-8ff812b1eca2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:18.262076 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d009a83-b6c2-4ce1-a04f-8ff812b1eca2 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:18.269525 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 985/1969] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:14:18 2022] PUT /identity/v3/projects/9221b61786004e0b8e21b13c18490fed/users/d56b1b3d27c744858093f3ab829b8c81/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:18.279689 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910d39b-0195-41ef-9e70-155db7a3c1d3 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:18.280021 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910d39b-0195-41ef-9e70-155db7a3c1d3 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:18.280298 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910d39b-0195-41ef-9e70-155db7a3c1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:18.321214 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-9910d39b-0195-41ef-9e70-155db7a3c1d3 None None] MFA Rules not processed for user `d56b1b3d27c744858093f3ab829b8c81`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:18.322690 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-9910d39b-0195-41ef-9e70-155db7a3c1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:18.329798 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 985/1970] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:14:18 2022] POST /identity/v3/projects => generated 385 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:14:18.334190 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d009a83-b6c2-4ce1-a04f-8ff812b1eca2 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:18.441091 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 986/1971] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:14:18.443929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d009a83-b6c2-4ce1-a04f-8ff812b1eca2 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:18.449126 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910d39b-0195-41ef-9e70-155db7a3c1d3 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:18.451208 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-9910d39b-0195-41ef-9e70-155db7a3c1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:18.453824 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195d803-c1b0-483e-a9e4-060672a929fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:18.454655 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195d803-c1b0-483e-a9e4-060672a929fa admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:18.454985 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195d803-c1b0-483e-a9e4-060672a929fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:18.455262 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195d803-c1b0-483e-a9e4-060672a929fa admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:18.455903 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b195d803-c1b0-483e-a9e4-060672a929fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:18.457999 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b195d803-c1b0-483e-a9e4-060672a929fa admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:18.491675 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 986/1972] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:14:18 2022] POST /identity/v3/users => generated 293 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:14:18.494891 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195d803-c1b0-483e-a9e4-060672a929fa admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:18.507839 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195d803-c1b0-483e-a9e4-060672a929fa admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:18.516187 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a9a99-33fb-4e68-9c58-068e22015646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:18.516874 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a9a99-33fb-4e68-9c58-068e22015646 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:18.517271 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a9a99-33fb-4e68-9c58-068e22015646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:18.517557 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a9a99-33fb-4e68-9c58-068e22015646 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:18.518271 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3a9a99-33fb-4e68-9c58-068e22015646 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:18.519959 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3a9a99-33fb-4e68-9c58-068e22015646 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:18.525644 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910d39b-0195-41ef-9e70-155db7a3c1d3 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:18.532451 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 987/1973] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:14:18.536382 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a9a99-33fb-4e68-9c58-068e22015646 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:18.547188 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a9a99-33fb-4e68-9c58-068e22015646 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:18.551946 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e8d49-56f8-4dda-b3c0-267ea7d703a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:18.552672 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e8d49-56f8-4dda-b3c0-267ea7d703a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:18.552953 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e8d49-56f8-4dda-b3c0-267ea7d703a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:18.553223 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e8d49-56f8-4dda-b3c0-267ea7d703a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:18.553860 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93e8d49-56f8-4dda-b3c0-267ea7d703a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:18.555490 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93e8d49-56f8-4dda-b3c0-267ea7d703a8 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:18.566523 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 988/1974] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:14:18.570550 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e8d49-56f8-4dda-b3c0-267ea7d703a8 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:18.582124 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e8d49-56f8-4dda-b3c0-267ea7d703a8 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:18.589182 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55cb43-9c5c-4cc4-8161-ff59527c09c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:18.589960 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55cb43-9c5c-4cc4-8161-ff59527c09c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:18.590308 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55cb43-9c5c-4cc4-8161-ff59527c09c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:18.590580 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55cb43-9c5c-4cc4-8161-ff59527c09c7 admin admin] PATH_INFO: `/v3/projects/0aef64154651444f9962020581d958db/users/8886ddd9b77244578be5d25cad17fb95/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:18.591888 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f002aa46-b35a-4eff-992b-e72c45d32453 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:18.592584 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002aa46-b35a-4eff-992b-e72c45d32453 service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:18.592981 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002aa46-b35a-4eff-992b-e72c45d32453 service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:18.594101 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002aa46-b35a-4eff-992b-e72c45d32453 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:18.609647 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a55cb43-9c5c-4cc4-8161-ff59527c09c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0aef64154651444f9962020581d958db, user_id=8886ddd9b77244578be5d25cad17fb95, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:18.612480 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a55cb43-9c5c-4cc4-8161-ff59527c09c7 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:18.616981 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:18.623027 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:18.630042 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 989/1975] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:14:18 2022] PUT /identity/v3/projects/0aef64154651444f9962020581d958db/users/8886ddd9b77244578be5d25cad17fb95/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:14:18.641331 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa80f7-e4ff-488a-bb66-d8ac6eac26d1 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:18.641619 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa80f7-e4ff-488a-bb66-d8ac6eac26d1 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:18.641888 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa80f7-e4ff-488a-bb66-d8ac6eac26d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:18.673274 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-90aa80f7-e4ff-488a-bb66-d8ac6eac26d1 None None] MFA Rules not processed for user `8886ddd9b77244578be5d25cad17fb95`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:18.674546 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-90aa80f7-e4ff-488a-bb66-d8ac6eac26d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:18.698717 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f002aa46-b35a-4eff-992b-e72c45d32453 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:18.700205 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f002aa46-b35a-4eff-992b-e72c45d32453 service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:18.712375 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 987/1976] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:14:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:14:18.737385 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f002aa46-b35a-4eff-992b-e72c45d32453 service cinder] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:18.750452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f002aa46-b35a-4eff-992b-e72c45d32453 service cinder] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:18.755120 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 990/1977] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:14:18.760228 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa80f7-e4ff-488a-bb66-d8ac6eac26d1 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:18.821194 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788feaa-592b-4f25-b18d-5645669a2aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:18.822032 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788feaa-592b-4f25-b18d-5645669a2aec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:18.822229 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788feaa-592b-4f25-b18d-5645669a2aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:18.822581 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788feaa-592b-4f25-b18d-5645669a2aec admin admin] PATH_INFO: `/v3/users/d56b1b3d27c744858093f3ab829b8c81` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:18.823849 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1788feaa-592b-4f25-b18d-5645669a2aec admin admin] RBAC: Authorizing `identity:delete_user(user_id=d56b1b3d27c744858093f3ab829b8c81)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:18.825857 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1788feaa-592b-4f25-b18d-5645669a2aec admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:18.837822 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa80f7-e4ff-488a-bb66-d8ac6eac26d1 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:18.843617 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714fede-5f27-4b50-bbca-81e9cb9543e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:18.844230 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714fede-5f27-4b50-bbca-81e9cb9543e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:18.844530 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714fede-5f27-4b50-bbca-81e9cb9543e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:18.844788 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714fede-5f27-4b50-bbca-81e9cb9543e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:18.845669 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714fede-5f27-4b50-bbca-81e9cb9543e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:18.852453 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714fede-5f27-4b50-bbca-81e9cb9543e5 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:18.890235 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 991/1978] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:14:18 2022] POST /identity/v3/projects => generated 385 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:14:18.890451 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-1788feaa-592b-4f25-b18d-5645669a2aec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd56b1b3d27c744858093f3ab829b8c81'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:18.891725 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714fede-5f27-4b50-bbca-81e9cb9543e5 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:18.897207 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-1788feaa-592b-4f25-b18d-5645669a2aec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd56b1b3d27c744858093f3ab829b8c81'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:18.898117 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-1788feaa-592b-4f25-b18d-5645669a2aec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd56b1b3d27c744858093f3ab829b8c81'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:18.898936 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-1788feaa-592b-4f25-b18d-5645669a2aec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd56b1b3d27c744858093f3ab829b8c81'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:18.909223 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-1788feaa-592b-4f25-b18d-5645669a2aec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd56b1b3d27c744858093f3ab829b8c81'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:18.930379 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 988/1979] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:14:18 2022] DELETE /identity/v3/users/d56b1b3d27c744858093f3ab829b8c81 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:18.934988 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b11fe-8272-4500-a54c-4c94f128677c None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:18.936110 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b11fe-8272-4500-a54c-4c94f128677c None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:18.936397 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b11fe-8272-4500-a54c-4c94f128677c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:18.960576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714fede-5f27-4b50-bbca-81e9cb9543e5 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:18.967285 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-d82b11fe-8272-4500-a54c-4c94f128677c None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:18.969287 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-d82b11fe-8272-4500-a54c-4c94f128677c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:18.982603 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:19.016038 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:19.029232 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb801227-0769-4dd3-8d3d-1b6bfa0ab2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:19.029867 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb801227-0769-4dd3-8d3d-1b6bfa0ab2ef admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:19.030140 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb801227-0769-4dd3-8d3d-1b6bfa0ab2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:19.030418 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb801227-0769-4dd3-8d3d-1b6bfa0ab2ef admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:19.031775 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb801227-0769-4dd3-8d3d-1b6bfa0ab2ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:19.034211 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb801227-0769-4dd3-8d3d-1b6bfa0ab2ef admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:19.050166 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 989/1980] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:14:19.060744 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b11fe-8272-4500-a54c-4c94f128677c None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:19.064817 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 992/1981] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:14:18 2022] POST /identity/v3/users => generated 299 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:14:19.064977 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-d82b11fe-8272-4500-a54c-4c94f128677c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:19.066148 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb801227-0769-4dd3-8d3d-1b6bfa0ab2ef admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:19.067835 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-eb801227-0769-4dd3-8d3d-1b6bfa0ab2ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:19.080634 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb801227-0769-4dd3-8d3d-1b6bfa0ab2ef admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:19.090758 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df3623c-9ef4-4e40-b9b9-e02da90d9f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:19.091647 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3623c-9ef4-4e40-b9b9-e02da90d9f70 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:19.091929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3623c-9ef4-4e40-b9b9-e02da90d9f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:19.092275 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3623c-9ef4-4e40-b9b9-e02da90d9f70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:19.094077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df3623c-9ef4-4e40-b9b9-e02da90d9f70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:19.096481 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df3623c-9ef4-4e40-b9b9-e02da90d9f70 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:19.109695 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 993/1982] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:14:19.114428 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df3623c-9ef4-4e40-b9b9-e02da90d9f70 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:19.141241 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df3623c-9ef4-4e40-b9b9-e02da90d9f70 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:19.148807 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f738563d-bcdc-4844-858b-848c4b20248b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:19.149858 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738563d-bcdc-4844-858b-848c4b20248b admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:19.150372 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738563d-bcdc-4844-858b-848c4b20248b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:19.151317 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738563d-bcdc-4844-858b-848c4b20248b admin admin] PATH_INFO: `/v3/projects/024a41a21a3449ffb2c4301b287686bb/users/3609dd077d69452dae8cbc1bd642d78b/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:19.151552 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b11fe-8272-4500-a54c-4c94f128677c None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:19.169690 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf952a-1b6c-45b7-8c3e-c54a38b3f2d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:19.170690 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf952a-1b6c-45b7-8c3e-c54a38b3f2d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:19.171000 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf952a-1b6c-45b7-8c3e-c54a38b3f2d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:19.172833 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf952a-1b6c-45b7-8c3e-c54a38b3f2d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:19.175083 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f738563d-bcdc-4844-858b-848c4b20248b admin admin] RBAC: Authorizing `identity:create_grant(project_id=024a41a21a3449ffb2c4301b287686bb, user_id=3609dd077d69452dae8cbc1bd642d78b, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:19.179237 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f738563d-bcdc-4844-858b-848c4b20248b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:19.191317 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bf952a-1b6c-45b7-8c3e-c54a38b3f2d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:19.192871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bf952a-1b6c-45b7-8c3e-c54a38b3f2d1 service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:19.209838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 990/1984] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:14:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:14:19.210039 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 994/1984] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:14:19 2022] PUT /identity/v3/projects/024a41a21a3449ffb2c4301b287686bb/users/3609dd077d69452dae8cbc1bd642d78b/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:19.210976 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf952a-1b6c-45b7-8c3e-c54a38b3f2d1 service neutron] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:19.221220 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf952a-1b6c-45b7-8c3e-c54a38b3f2d1 service neutron] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:19.281207 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a5078-f16e-4801-a8c3-68298226182c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:19.281486 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a5078-f16e-4801-a8c3-68298226182c admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:19.281749 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a5078-f16e-4801-a8c3-68298226182c admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:19.282027 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a5078-f16e-4801-a8c3-68298226182c admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:19.282673 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a5078-f16e-4801-a8c3-68298226182c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:19.284394 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a5078-f16e-4801-a8c3-68298226182c admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:19.298997 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 991/1985] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:14:19.300399 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f738563d-bcdc-4844-858b-848c4b20248b admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:19.312690 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f738563d-bcdc-4844-858b-848c4b20248b admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:19.318301 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdccd393-c438-4ef2-9a44-48f89fe4f938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:19.319007 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccd393-c438-4ef2-9a44-48f89fe4f938 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:19.319260 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccd393-c438-4ef2-9a44-48f89fe4f938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:19.319725 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccd393-c438-4ef2-9a44-48f89fe4f938 admin admin] PATH_INFO: `/v3/projects/024a41a21a3449ffb2c4301b287686bb/users/3609dd077d69452dae8cbc1bd642d78b/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:19.321623 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdccd393-c438-4ef2-9a44-48f89fe4f938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=024a41a21a3449ffb2c4301b287686bb, user_id=3609dd077d69452dae8cbc1bd642d78b, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:19.324345 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdccd393-c438-4ef2-9a44-48f89fe4f938 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:19.335687 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 995/1986] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:14:19 2022] PUT /identity/v3/projects/024a41a21a3449ffb2c4301b287686bb/users/3609dd077d69452dae8cbc1bd642d78b/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:19.346815 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b06513-08ae-4fcc-901f-39f11bfe40ea None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:19.347149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b06513-08ae-4fcc-901f-39f11bfe40ea None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:19.347527 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b06513-08ae-4fcc-901f-39f11bfe40ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:19.376970 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-f6b06513-08ae-4fcc-901f-39f11bfe40ea None None] MFA Rules not processed for user `3609dd077d69452dae8cbc1bd642d78b`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:19.378238 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-f6b06513-08ae-4fcc-901f-39f11bfe40ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:19.457224 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 992/1987] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:19 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:19.469094 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdccd393-c438-4ef2-9a44-48f89fe4f938 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:19.470829 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-bdccd393-c438-4ef2-9a44-48f89fe4f938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:19.537880 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdccd393-c438-4ef2-9a44-48f89fe4f938 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:19.540394 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b06513-08ae-4fcc-901f-39f11bfe40ea None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:19.542127 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-f6b06513-08ae-4fcc-901f-39f11bfe40ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:19.542792 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda57713-aa9e-4db4-a846-939ac958975d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:19.543692 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda57713-aa9e-4db4-a846-939ac958975d service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:19.544070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda57713-aa9e-4db4-a846-939ac958975d service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:19.544437 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda57713-aa9e-4db4-a846-939ac958975d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:19.546676 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-eda57713-aa9e-4db4-a846-939ac958975d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:19.550365 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:19.643472 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b06513-08ae-4fcc-901f-39f11bfe40ea None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:19.649112 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ab7e3-ccea-4ec0-9e54-8e0c642be050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:19.649843 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ab7e3-ccea-4ec0-9e54-8e0c642be050 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:19.650222 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ab7e3-ccea-4ec0-9e54-8e0c642be050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:19.650609 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ab7e3-ccea-4ec0-9e54-8e0c642be050 admin admin] PATH_INFO: `/v3/projects/9221b61786004e0b8e21b13c18490fed` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:19.652366 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ab7e3-ccea-4ec0-9e54-8e0c642be050 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9221b61786004e0b8e21b13c18490fed)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:19.656676 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ab7e3-ccea-4ec0-9e54-8e0c642be050 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:19.661395 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda57713-aa9e-4db4-a846-939ac958975d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:19.672440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda57713-aa9e-4db4-a846-939ac958975d service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:19.686683 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 996/1988] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:14:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:14:19.719927 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-bb2ab7e3-ccea-4ec0-9e54-8e0c642be050 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9221b61786004e0b8e21b13c18490fed', 'request_id': 'req-bb2ab7e3-ccea-4ec0-9e54-8e0c642be050'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:19.726711 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-bb2ab7e3-ccea-4ec0-9e54-8e0c642be050 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9221b61786004e0b8e21b13c18490fed', 'request_id': 'req-bb2ab7e3-ccea-4ec0-9e54-8e0c642be050'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:19.740036 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-bb2ab7e3-ccea-4ec0-9e54-8e0c642be050 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9221b61786004e0b8e21b13c18490fed', 'request_id': 'req-bb2ab7e3-ccea-4ec0-9e54-8e0c642be050'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:19.760201 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-bb2ab7e3-ccea-4ec0-9e54-8e0c642be050 admin admin] The token cache is being invalidate because project 9221b61786004e0b8e21b13c18490fed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:14:19.760666 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-bb2ab7e3-ccea-4ec0-9e54-8e0c642be050 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:19.761442 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-bb2ab7e3-ccea-4ec0-9e54-8e0c642be050 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:19.767056 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 993/1989] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:14:19 2022] DELETE /identity/v3/projects/9221b61786004e0b8e21b13c18490fed => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:14:19.768717 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b9c56-44fd-42b6-98c8-edf2bc873ac4 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:19.769008 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b9c56-44fd-42b6-98c8-edf2bc873ac4 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:19.769314 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b9c56-44fd-42b6-98c8-edf2bc873ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:19.791774 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-e11b9c56-44fd-42b6-98c8-edf2bc873ac4 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:19.793026 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e11b9c56-44fd-42b6-98c8-edf2bc873ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:19.803228 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:19.860992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 997/1990] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:19.868934 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa7a50-0e56-4815-8dfc-5a5b3f599d88 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:19.869364 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa7a50-0e56-4815-8dfc-5a5b3f599d88 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:19.869737 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa7a50-0e56-4815-8dfc-5a5b3f599d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:19.892490 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b9c56-44fd-42b6-98c8-edf2bc873ac4 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:19.893965 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e11b9c56-44fd-42b6-98c8-edf2bc873ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:19.895630 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-67aa7a50-0e56-4815-8dfc-5a5b3f599d88 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:19.897087 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-67aa7a50-0e56-4815-8dfc-5a5b3f599d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:19.906881 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b9c56-44fd-42b6-98c8-edf2bc873ac4 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:19.910659 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 994/1991] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:19.911411 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa7a50-0e56-4815-8dfc-5a5b3f599d88 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:19.915692 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf70d93-b042-45b8-8de9-ac67831baf27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:19.916336 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf70d93-b042-45b8-8de9-ac67831baf27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:19.916627 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf70d93-b042-45b8-8de9-ac67831baf27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:19.917000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf70d93-b042-45b8-8de9-ac67831baf27 admin admin] PATH_INFO: `/v3/users/658aeed9964b4b33b9396eca04cffadb` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:19.918310 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf70d93-b042-45b8-8de9-ac67831baf27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=658aeed9964b4b33b9396eca04cffadb)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:19.920430 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf70d93-b042-45b8-8de9-ac67831baf27 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:19.922050 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa7a50-0e56-4815-8dfc-5a5b3f599d88 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:19.930740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d3eec-25c8-4564-ac38-4f9ace668b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:19.931516 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d3eec-25c8-4564-ac38-4f9ace668b00 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:19.931924 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d3eec-25c8-4564-ac38-4f9ace668b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:19.932379 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d3eec-25c8-4564-ac38-4f9ace668b00 admin admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:19.933094 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d3eec-25c8-4564-ac38-4f9ace668b00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:14:19.933417 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d3eec-25c8-4564-ac38-4f9ace668b00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:19.935014 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d3eec-25c8-4564-ac38-4f9ace668b00 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:19.951424 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 995/1992] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:14:19.956766 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d3eec-25c8-4564-ac38-4f9ace668b00 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:19.976437 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d3eec-25c8-4564-ac38-4f9ace668b00 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:19.986608 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e5038-20b1-4cdc-bfc3-cc1400efa347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:19.987298 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e5038-20b1-4cdc-bfc3-cc1400efa347 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:19.987579 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e5038-20b1-4cdc-bfc3-cc1400efa347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:19.987850 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e5038-20b1-4cdc-bfc3-cc1400efa347 admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:19.988530 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8e5038-20b1-4cdc-bfc3-cc1400efa347 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:19.988901 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-caf70d93-b042-45b8-8de9-ac67831baf27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '658aeed9964b4b33b9396eca04cffadb'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:19.990476 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8e5038-20b1-4cdc-bfc3-cc1400efa347 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:20.001752 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-caf70d93-b042-45b8-8de9-ac67831baf27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '658aeed9964b4b33b9396eca04cffadb'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:20.003246 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-caf70d93-b042-45b8-8de9-ac67831baf27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '658aeed9964b4b33b9396eca04cffadb'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:20.004196 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-caf70d93-b042-45b8-8de9-ac67831baf27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '658aeed9964b4b33b9396eca04cffadb'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:20.013963 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-caf70d93-b042-45b8-8de9-ac67831baf27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '658aeed9964b4b33b9396eca04cffadb'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:20.034209 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 996/1993] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:14:19 2022] POST /identity/v3/projects => generated 375 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:14:20.035373 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e5038-20b1-4cdc-bfc3-cc1400efa347 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:20.037399 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-5d8e5038-20b1-4cdc-bfc3-cc1400efa347 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:20.044158 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 998/1994] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:14:19 2022] DELETE /identity/v3/users/658aeed9964b4b33b9396eca04cffadb => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:14:20.056759 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f05a92-bc60-414c-aff8-d5b53d5af496 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:20.057053 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f05a92-bc60-414c-aff8-d5b53d5af496 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:20.057330 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f05a92-bc60-414c-aff8-d5b53d5af496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:20.095563 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-d9f05a92-bc60-414c-aff8-d5b53d5af496 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:20.097841 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d9f05a92-bc60-414c-aff8-d5b53d5af496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:20.132483 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:20.156164 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e5038-20b1-4cdc-bfc3-cc1400efa347 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:20.210760 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 999/1995] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:20.221938 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f05a92-bc60-414c-aff8-d5b53d5af496 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:20.223867 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d9f05a92-bc60-414c-aff8-d5b53d5af496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:20.224755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f60bb-9b28-4ce0-b647-fff95efd546a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:20.225663 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f60bb-9b28-4ce0-b647-fff95efd546a admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:20.225946 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f60bb-9b28-4ce0-b647-fff95efd546a admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:20.226216 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f60bb-9b28-4ce0-b647-fff95efd546a admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:20.226904 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3f60bb-9b28-4ce0-b647-fff95efd546a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:20.228935 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3f60bb-9b28-4ce0-b647-fff95efd546a admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:20.257784 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 997/1996] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:14:20 2022] POST /identity/v3/users => generated 288 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:14:20.259403 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f60bb-9b28-4ce0-b647-fff95efd546a admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:20.262874 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:20.270188 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f60bb-9b28-4ce0-b647-fff95efd546a admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:20.275276 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06552646-aed5-4eac-9cc0-d0ba86fea6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:20.276336 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06552646-aed5-4eac-9cc0-d0ba86fea6fd admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:20.276618 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06552646-aed5-4eac-9cc0-d0ba86fea6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:20.276931 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06552646-aed5-4eac-9cc0-d0ba86fea6fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:20.278201 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06552646-aed5-4eac-9cc0-d0ba86fea6fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:20.281344 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06552646-aed5-4eac-9cc0-d0ba86fea6fd admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:20.293098 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 998/1997] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:14:20.302407 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06552646-aed5-4eac-9cc0-d0ba86fea6fd admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:20.307216 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f05a92-bc60-414c-aff8-d5b53d5af496 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:20.311221 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af479b5a-fbc5-4b57-b8e3-b8eed08941af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:20.313515 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af479b5a-fbc5-4b57-b8e3-b8eed08941af service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:20.313631 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06552646-aed5-4eac-9cc0-d0ba86fea6fd admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:20.313802 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af479b5a-fbc5-4b57-b8e3-b8eed08941af service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:20.314322 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af479b5a-fbc5-4b57-b8e3-b8eed08941af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:20.318393 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531bf881-1939-46e0-aea6-5b2d64007045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:20.319057 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bf881-1939-46e0-aea6-5b2d64007045 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:20.319334 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bf881-1939-46e0-aea6-5b2d64007045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:20.319601 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bf881-1939-46e0-aea6-5b2d64007045 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:20.320242 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531bf881-1939-46e0-aea6-5b2d64007045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:20.321931 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531bf881-1939-46e0-aea6-5b2d64007045 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:20.323913 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af479b5a-fbc5-4b57-b8e3-b8eed08941af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:20.325428 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af479b5a-fbc5-4b57-b8e3-b8eed08941af service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:20.336783 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 999/1998] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:14:20.336934 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1000/1999] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:14:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:14:20.338552 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531bf881-1939-46e0-aea6-5b2d64007045 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:20.351333 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531bf881-1939-46e0-aea6-5b2d64007045 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:20.356061 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a66bcf-ffc1-452b-897f-d1d2de8635ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:20.356729 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a66bcf-ffc1-452b-897f-d1d2de8635ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:20.357007 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a66bcf-ffc1-452b-897f-d1d2de8635ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:20.357279 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a66bcf-ffc1-452b-897f-d1d2de8635ca admin admin] PATH_INFO: `/v3/projects/48723e2b611247819f4c1ea39d91be23/users/84e22405b541490f8ce26de7ae73acd6/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:20.375032 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a66bcf-ffc1-452b-897f-d1d2de8635ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=48723e2b611247819f4c1ea39d91be23, user_id=84e22405b541490f8ce26de7ae73acd6, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:20.378186 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a66bcf-ffc1-452b-897f-d1d2de8635ca admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:20.394430 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1000/2000] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:14:20 2022] PUT /identity/v3/projects/48723e2b611247819f4c1ea39d91be23/users/84e22405b541490f8ce26de7ae73acd6/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:14:20.406921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36595df3-cbbf-4538-9024-304acf7d46cd None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:20.407436 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36595df3-cbbf-4538-9024-304acf7d46cd None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:20.408200 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36595df3-cbbf-4538-9024-304acf7d46cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:20.449763 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-36595df3-cbbf-4538-9024-304acf7d46cd None None] MFA Rules not processed for user `84e22405b541490f8ce26de7ae73acd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:20.450730 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-36595df3-cbbf-4538-9024-304acf7d46cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:20.547843 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1001/2001] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:14:20.552551 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a66bcf-ffc1-452b-897f-d1d2de8635ca admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:20.629330 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a66bcf-ffc1-452b-897f-d1d2de8635ca admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:20.634106 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79503b-1d59-4e43-848f-a921c18d24e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:20.634730 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79503b-1d59-4e43-848f-a921c18d24e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:20.635090 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79503b-1d59-4e43-848f-a921c18d24e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:20.635370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79503b-1d59-4e43-848f-a921c18d24e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:20.635994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c79503b-1d59-4e43-848f-a921c18d24e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:20.637898 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c79503b-1d59-4e43-848f-a921c18d24e4 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:20.672076 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1001/2002] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:14:20 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:14:20.673306 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36595df3-cbbf-4538-9024-304acf7d46cd None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:20.686182 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36595df3-cbbf-4538-9024-304acf7d46cd None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:20.754425 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75233b0-1715-4a69-9e44-8de2a6caf3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:20.755412 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75233b0-1715-4a69-9e44-8de2a6caf3ef admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:20.755680 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75233b0-1715-4a69-9e44-8de2a6caf3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:20.755941 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75233b0-1715-4a69-9e44-8de2a6caf3ef admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:20.756611 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75233b0-1715-4a69-9e44-8de2a6caf3ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:20.759858 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75233b0-1715-4a69-9e44-8de2a6caf3ef admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:20.787462 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1002/2003] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:14:20 2022] POST /identity/v3/users => generated 293 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:14:20.791792 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79503b-1d59-4e43-848f-a921c18d24e4 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:20.804755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79503b-1d59-4e43-848f-a921c18d24e4 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:20.814768 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841b0a8-9059-48dd-9666-d2b7dda60894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:20.815496 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841b0a8-9059-48dd-9666-d2b7dda60894 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:20.815776 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841b0a8-9059-48dd-9666-d2b7dda60894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:20.816402 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841b0a8-9059-48dd-9666-d2b7dda60894 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:20.817196 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e841b0a8-9059-48dd-9666-d2b7dda60894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:20.818797 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e841b0a8-9059-48dd-9666-d2b7dda60894 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:20.831217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1002/2004] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:14:20.840274 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75233b0-1715-4a69-9e44-8de2a6caf3ef admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:20.851439 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75233b0-1715-4a69-9e44-8de2a6caf3ef admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:20.856169 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e768c5d4-56fb-4268-878d-99bb6234b41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:20.856985 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768c5d4-56fb-4268-878d-99bb6234b41a admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:20.857265 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768c5d4-56fb-4268-878d-99bb6234b41a admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:20.857545 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768c5d4-56fb-4268-878d-99bb6234b41a admin admin] PATH_INFO: `/v3/projects/d07dc79346cb492fa539154818fc36eb/users/702a4174900c4701896578cd2892f336/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:20.876954 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e768c5d4-56fb-4268-878d-99bb6234b41a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d07dc79346cb492fa539154818fc36eb, user_id=702a4174900c4701896578cd2892f336, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:20.883185 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e768c5d4-56fb-4268-878d-99bb6234b41a admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:20.902337 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1003/2005] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:14:20 2022] PUT /identity/v3/projects/d07dc79346cb492fa539154818fc36eb/users/702a4174900c4701896578cd2892f336/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:14:20.906599 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841b0a8-9059-48dd-9666-d2b7dda60894 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:20.918623 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841b0a8-9059-48dd-9666-d2b7dda60894 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:20.935451 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:20.991792 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e768c5d4-56fb-4268-878d-99bb6234b41a admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:20.993807 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e768c5d4-56fb-4268-878d-99bb6234b41a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:21.033389 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576c344-e9b4-464b-9ce9-d0e658734a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:21.034900 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:21.035091 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576c344-e9b4-464b-9ce9-d0e658734a86 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:21.035535 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576c344-e9b4-464b-9ce9-d0e658734a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:21.036065 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576c344-e9b4-464b-9ce9-d0e658734a86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:21.037522 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e576c344-e9b4-464b-9ce9-d0e658734a86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:21.039679 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e576c344-e9b4-464b-9ce9-d0e658734a86 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:21.053405 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1003/2006] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:14:21.059149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576c344-e9b4-464b-9ce9-d0e658734a86 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:21.079151 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576c344-e9b4-464b-9ce9-d0e658734a86 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:21.079930 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03745c29-c650-43a2-984a-661c81801915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:21.082721 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03745c29-c650-43a2-984a-661c81801915 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:21.082721 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03745c29-c650-43a2-984a-661c81801915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:21.082721 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03745c29-c650-43a2-984a-661c81801915 admin admin] PATH_INFO: `/v3/projects/d07dc79346cb492fa539154818fc36eb/users/702a4174900c4701896578cd2892f336/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:21.083791 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03745c29-c650-43a2-984a-661c81801915 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d07dc79346cb492fa539154818fc36eb, user_id=702a4174900c4701896578cd2892f336, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:21.087085 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03745c29-c650-43a2-984a-661c81801915 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:21.089984 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e768c5d4-56fb-4268-878d-99bb6234b41a admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:21.095883 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf2e3b-7c9a-4f9e-900a-9cc1b12867b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:21.096501 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf2e3b-7c9a-4f9e-900a-9cc1b12867b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:21.096776 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf2e3b-7c9a-4f9e-900a-9cc1b12867b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:21.097047 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf2e3b-7c9a-4f9e-900a-9cc1b12867b2 admin admin] PATH_INFO: `/v3/projects/d8e5da90a062489d9f68dd6addbfd7a1` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:21.098299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf2e3b-7c9a-4f9e-900a-9cc1b12867b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8e5da90a062489d9f68dd6addbfd7a1)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:21.099377 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1004/2007] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:14:21 2022] PUT /identity/v3/projects/d07dc79346cb492fa539154818fc36eb/users/702a4174900c4701896578cd2892f336/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:21.104034 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf2e3b-7c9a-4f9e-900a-9cc1b12867b2 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:21.109721 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c97fac-3a76-466b-ac23-992033581080 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:21.110000 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c97fac-3a76-466b-ac23-992033581080 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:21.110267 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c97fac-3a76-466b-ac23-992033581080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:21.151457 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-c8c97fac-3a76-466b-ac23-992033581080 None None] MFA Rules not processed for user `702a4174900c4701896578cd2892f336`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:21.152669 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-c8c97fac-3a76-466b-ac23-992033581080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:21.162786 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-94bf2e3b-7c9a-4f9e-900a-9cc1b12867b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8e5da90a062489d9f68dd6addbfd7a1', 'request_id': 'req-94bf2e3b-7c9a-4f9e-900a-9cc1b12867b2'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:21.179344 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-94bf2e3b-7c9a-4f9e-900a-9cc1b12867b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8e5da90a062489d9f68dd6addbfd7a1', 'request_id': 'req-94bf2e3b-7c9a-4f9e-900a-9cc1b12867b2'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:21.186595 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-94bf2e3b-7c9a-4f9e-900a-9cc1b12867b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8e5da90a062489d9f68dd6addbfd7a1', 'request_id': 'req-94bf2e3b-7c9a-4f9e-900a-9cc1b12867b2'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:21.198598 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-94bf2e3b-7c9a-4f9e-900a-9cc1b12867b2 admin admin] The token cache is being invalidate because project d8e5da90a062489d9f68dd6addbfd7a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:14:21.198993 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-94bf2e3b-7c9a-4f9e-900a-9cc1b12867b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:21.199765 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-94bf2e3b-7c9a-4f9e-900a-9cc1b12867b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:21.202568 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1004/2008] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:14:20 2022] DELETE /identity/v3/projects/d8e5da90a062489d9f68dd6addbfd7a1 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:14:21.207419 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef24a5-453f-46f0-ae73-3e74391b67f2 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:21.207704 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef24a5-453f-46f0-ae73-3e74391b67f2 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:21.207968 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef24a5-453f-46f0-ae73-3e74391b67f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:21.229830 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-84ef24a5-453f-46f0-ae73-3e74391b67f2 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:21.231263 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1005/2009] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:21 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:14:21.231397 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-84ef24a5-453f-46f0-ae73-3e74391b67f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:21.243736 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c97fac-3a76-466b-ac23-992033581080 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:21.245322 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-c8c97fac-3a76-466b-ac23-992033581080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:21.306575 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c97fac-3a76-466b-ac23-992033581080 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:21.307281 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1005/2010] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:21.308122 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef24a5-453f-46f0-ae73-3e74391b67f2 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:21.309399 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-84ef24a5-453f-46f0-ae73-3e74391b67f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:21.317841 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eea535-748a-4f6a-9fa9-427fe8ac540f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:21.318459 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eea535-748a-4f6a-9fa9-427fe8ac540f service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:21.318733 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eea535-748a-4f6a-9fa9-427fe8ac540f service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:21.319029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eea535-748a-4f6a-9fa9-427fe8ac540f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:21.320791 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-e9eea535-748a-4f6a-9fa9-427fe8ac540f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:21.322250 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef24a5-453f-46f0-ae73-3e74391b67f2 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:21.326014 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10edccd0-7632-4f2d-a8aa-1f8ca1dc2bb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:21.326607 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10edccd0-7632-4f2d-a8aa-1f8ca1dc2bb1 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:21.327432 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10edccd0-7632-4f2d-a8aa-1f8ca1dc2bb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:21.327708 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10edccd0-7632-4f2d-a8aa-1f8ca1dc2bb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:21.329400 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-10edccd0-7632-4f2d-a8aa-1f8ca1dc2bb1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:21.361371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:21.383440 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eea535-748a-4f6a-9fa9-427fe8ac540f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:21.384961 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eea535-748a-4f6a-9fa9-427fe8ac540f service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:21.386365 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:21.389015 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10edccd0-7632-4f2d-a8aa-1f8ca1dc2bb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:21.390807 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10edccd0-7632-4f2d-a8aa-1f8ca1dc2bb1 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:21.394468 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1006/2011] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:14:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:14:21.395744 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9d9af-cbd0-4bf2-a424-2bdac6e4b9fe None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:21.396717 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9d9af-cbd0-4bf2-a424-2bdac6e4b9fe None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:21.396717 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9d9af-cbd0-4bf2-a424-2bdac6e4b9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:21.407228 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1006/2012] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:14:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:14:21.418080 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-5ae9d9af-cbd0-4bf2-a424-2bdac6e4b9fe None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:21.419396 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-5ae9d9af-cbd0-4bf2-a424-2bdac6e4b9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:21.432371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1007/2013] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:21.433689 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10edccd0-7632-4f2d-a8aa-1f8ca1dc2bb1 service cinder] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:21.445965 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10edccd0-7632-4f2d-a8aa-1f8ca1dc2bb1 service cinder] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:21.451064 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b3e325-41fb-4a21-9b5e-105c9309e4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:21.451790 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3e325-41fb-4a21-9b5e-105c9309e4de admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:21.452067 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3e325-41fb-4a21-9b5e-105c9309e4de admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:21.452517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3e325-41fb-4a21-9b5e-105c9309e4de admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:21.453120 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b3e325-41fb-4a21-9b5e-105c9309e4de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:14:21.453489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b3e325-41fb-4a21-9b5e-105c9309e4de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:21.455221 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b3e325-41fb-4a21-9b5e-105c9309e4de admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:21.469408 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1007/2014] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:14:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:14:21.474020 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae9d9af-cbd0-4bf2-a424-2bdac6e4b9fe None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:21.485570 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae9d9af-cbd0-4bf2-a424-2bdac6e4b9fe None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:21.491073 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951f35c-eb00-4a81-9833-8bbeef8ed2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:21.491763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951f35c-eb00-4a81-9833-8bbeef8ed2fa admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:21.492113 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951f35c-eb00-4a81-9833-8bbeef8ed2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:21.492447 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951f35c-eb00-4a81-9833-8bbeef8ed2fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:21.493228 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f951f35c-eb00-4a81-9833-8bbeef8ed2fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:21.495148 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f951f35c-eb00-4a81-9833-8bbeef8ed2fa admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:21.533170 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1008/2015] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:14:21 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:14:21.537912 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b3e325-41fb-4a21-9b5e-105c9309e4de admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:21.549137 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b3e325-41fb-4a21-9b5e-105c9309e4de admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:21.617089 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2118c-82fd-40b9-b1e2-a4c25e61cf30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:21.617833 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2118c-82fd-40b9-b1e2-a4c25e61cf30 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:21.618117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2118c-82fd-40b9-b1e2-a4c25e61cf30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:21.618388 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2118c-82fd-40b9-b1e2-a4c25e61cf30 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:21.619067 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c2118c-82fd-40b9-b1e2-a4c25e61cf30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:21.621120 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c2118c-82fd-40b9-b1e2-a4c25e61cf30 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:21.647652 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1008/2016] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:14:21 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:14:21.649211 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951f35c-eb00-4a81-9833-8bbeef8ed2fa admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:21.689433 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951f35c-eb00-4a81-9833-8bbeef8ed2fa admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:21.695694 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c144bd09-2ea6-49ea-bce0-171df3152911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:21.696354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144bd09-2ea6-49ea-bce0-171df3152911 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:21.696643 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144bd09-2ea6-49ea-bce0-171df3152911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:21.697032 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144bd09-2ea6-49ea-bce0-171df3152911 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:21.697527 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c144bd09-2ea6-49ea-bce0-171df3152911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:21.699595 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c144bd09-2ea6-49ea-bce0-171df3152911 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:21.714880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1009/2017] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:14:21.718700 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2118c-82fd-40b9-b1e2-a4c25e61cf30 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:21.729337 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2118c-82fd-40b9-b1e2-a4c25e61cf30 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:21.734954 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f463a219-e9cf-4de7-94b8-112ddbde7b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:21.735607 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463a219-e9cf-4de7-94b8-112ddbde7b2b admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:21.735885 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463a219-e9cf-4de7-94b8-112ddbde7b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:21.736255 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463a219-e9cf-4de7-94b8-112ddbde7b2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:21.736962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f463a219-e9cf-4de7-94b8-112ddbde7b2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:21.739281 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f463a219-e9cf-4de7-94b8-112ddbde7b2b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:21.752607 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1009/2018] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:14:21.757078 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c144bd09-2ea6-49ea-bce0-171df3152911 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:21.771201 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c144bd09-2ea6-49ea-bce0-171df3152911 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:21.776222 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934d34f-f95c-4989-a852-933ba4a5f3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:21.776876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934d34f-f95c-4989-a852-933ba4a5f3f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:21.777154 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934d34f-f95c-4989-a852-933ba4a5f3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:21.777502 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934d34f-f95c-4989-a852-933ba4a5f3f8 admin admin] PATH_INFO: `/v3/projects/bbcad1815c724322942cbc57420cc919/users/27a50a253b3843d99747f432aa0e2cf5/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:21.793535 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3934d34f-f95c-4989-a852-933ba4a5f3f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbcad1815c724322942cbc57420cc919, user_id=27a50a253b3843d99747f432aa0e2cf5, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:21.797039 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3934d34f-f95c-4989-a852-933ba4a5f3f8 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:21.812866 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1010/2019] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:14:21 2022] PUT /identity/v3/projects/bbcad1815c724322942cbc57420cc919/users/27a50a253b3843d99747f432aa0e2cf5/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:21.824153 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0076e0-3ded-4bba-9661-3e301bb3603f None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:21.824454 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0076e0-3ded-4bba-9661-3e301bb3603f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:21.824870 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0076e0-3ded-4bba-9661-3e301bb3603f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:21.855234 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-5f0076e0-3ded-4bba-9661-3e301bb3603f None None] MFA Rules not processed for user `27a50a253b3843d99747f432aa0e2cf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:21.856803 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-5f0076e0-3ded-4bba-9661-3e301bb3603f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:21.934441 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1010/2020] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:14:21.942658 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934d34f-f95c-4989-a852-933ba4a5f3f8 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:22.001916 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934d34f-f95c-4989-a852-933ba4a5f3f8 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:22.005476 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e7941-8669-4222-bc6d-1d942d4e3c3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:22.006074 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e7941-8669-4222-bc6d-1d942d4e3c3b service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:22.006346 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e7941-8669-4222-bc6d-1d942d4e3c3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:22.006613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e7941-8669-4222-bc6d-1d942d4e3c3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:22.015096 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e7941-8669-4222-bc6d-1d942d4e3c3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:22.016507 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e7941-8669-4222-bc6d-1d942d4e3c3b service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:22.025242 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1011/2021] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:14:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:14:22.043957 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0076e0-3ded-4bba-9661-3e301bb3603f None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:22.056357 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0076e0-3ded-4bba-9661-3e301bb3603f None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:22.060315 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d80d6ff-9503-41ec-b800-c7840641f654 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:22.061124 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80d6ff-9503-41ec-b800-c7840641f654 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:22.061494 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80d6ff-9503-41ec-b800-c7840641f654 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:22.061893 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80d6ff-9503-41ec-b800-c7840641f654 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:22.064362 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1d80d6ff-9503-41ec-b800-c7840641f654 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:22.121741 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:22.149830 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d80d6ff-9503-41ec-b800-c7840641f654 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:22.151642 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d80d6ff-9503-41ec-b800-c7840641f654 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:22.164724 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1011/2022] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:14:22 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:14:24.031358 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e7941-8669-4222-bc6d-1d942d4e3c3b service cinder] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:24.032957 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-fc6e7941-8669-4222-bc6d-1d942d4e3c3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:24.107836 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e7941-8669-4222-bc6d-1d942d4e3c3b service cinder] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:24.113006 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2affe05d-0b91-4298-a7e4-b9721e94017d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:24.113764 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2affe05d-0b91-4298-a7e4-b9721e94017d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:24.114082 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2affe05d-0b91-4298-a7e4-b9721e94017d admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:24.114355 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2affe05d-0b91-4298-a7e4-b9721e94017d admin admin] PATH_INFO: `/v3/users/84e22405b541490f8ce26de7ae73acd6` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:24.115492 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2affe05d-0b91-4298-a7e4-b9721e94017d admin admin] RBAC: Authorizing `identity:delete_user(user_id=84e22405b541490f8ce26de7ae73acd6)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:24.117520 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2affe05d-0b91-4298-a7e4-b9721e94017d admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:24.138819 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:24.165084 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-2affe05d-0b91-4298-a7e4-b9721e94017d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84e22405b541490f8ce26de7ae73acd6'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:24.172083 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-2affe05d-0b91-4298-a7e4-b9721e94017d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84e22405b541490f8ce26de7ae73acd6'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:24.172733 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-2affe05d-0b91-4298-a7e4-b9721e94017d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84e22405b541490f8ce26de7ae73acd6'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:24.173327 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-2affe05d-0b91-4298-a7e4-b9721e94017d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84e22405b541490f8ce26de7ae73acd6'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:24.182016 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-2affe05d-0b91-4298-a7e4-b9721e94017d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84e22405b541490f8ce26de7ae73acd6'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:24.200054 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1012/2023] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:14:24 2022] DELETE /identity/v3/users/84e22405b541490f8ce26de7ae73acd6 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:24.205002 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c0c69-7c73-46da-ab39-51b3173a4fe8 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:24.205497 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c0c69-7c73-46da-ab39-51b3173a4fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:24.205603 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c0c69-7c73-46da-ab39-51b3173a4fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:24.223954 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-7c9c0c69-7c73-46da-ab39-51b3173a4fe8 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:24.225149 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7c9c0c69-7c73-46da-ab39-51b3173a4fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:24.281652 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1012/2024] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:24.291297 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2affe05d-0b91-4298-a7e4-b9721e94017d admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:24.292687 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-2affe05d-0b91-4298-a7e4-b9721e94017d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:24.349720 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2affe05d-0b91-4298-a7e4-b9721e94017d admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:24.353284 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e4ddd-a6ff-498b-9618-07bd6397b3fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:24.353881 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e4ddd-a6ff-498b-9618-07bd6397b3fa service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:24.354156 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e4ddd-a6ff-498b-9618-07bd6397b3fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:24.354425 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e4ddd-a6ff-498b-9618-07bd6397b3fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:24.363381 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3e4ddd-a6ff-498b-9618-07bd6397b3fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:24.364801 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3e4ddd-a6ff-498b-9618-07bd6397b3fa service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:24.374073 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1013/2025] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:14:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:14:24.646671 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c0c69-7c73-46da-ab39-51b3173a4fe8 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:24.648380 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7c9c0c69-7c73-46da-ab39-51b3173a4fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:24.663774 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c0c69-7c73-46da-ab39-51b3173a4fe8 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:24.668846 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0555e-a503-484c-84e4-0edfadcdfb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:24.669493 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0555e-a503-484c-84e4-0edfadcdfb9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:24.669769 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0555e-a503-484c-84e4-0edfadcdfb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:24.670038 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0555e-a503-484c-84e4-0edfadcdfb9d admin admin] PATH_INFO: `/v3/projects/48723e2b611247819f4c1ea39d91be23` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:24.671793 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba0555e-a503-484c-84e4-0edfadcdfb9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=48723e2b611247819f4c1ea39d91be23)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:24.673812 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba0555e-a503-484c-84e4-0edfadcdfb9d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:24.740534 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-aba0555e-a503-484c-84e4-0edfadcdfb9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48723e2b611247819f4c1ea39d91be23', 'request_id': 'req-aba0555e-a503-484c-84e4-0edfadcdfb9d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:24.759499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-aba0555e-a503-484c-84e4-0edfadcdfb9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48723e2b611247819f4c1ea39d91be23', 'request_id': 'req-aba0555e-a503-484c-84e4-0edfadcdfb9d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:24.768268 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-aba0555e-a503-484c-84e4-0edfadcdfb9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48723e2b611247819f4c1ea39d91be23', 'request_id': 'req-aba0555e-a503-484c-84e4-0edfadcdfb9d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:24.782160 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-aba0555e-a503-484c-84e4-0edfadcdfb9d admin admin] The token cache is being invalidate because project 48723e2b611247819f4c1ea39d91be23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:14:24.782756 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-aba0555e-a503-484c-84e4-0edfadcdfb9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:24.783926 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-aba0555e-a503-484c-84e4-0edfadcdfb9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:24.785839 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1013/2026] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:14:24 2022] DELETE /identity/v3/projects/48723e2b611247819f4c1ea39d91be23 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:24.789608 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e4ddd-a6ff-498b-9618-07bd6397b3fa service neutron] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:24.800755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e4ddd-a6ff-498b-9618-07bd6397b3fa service neutron] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:24.861368 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3e4353-d0fb-43f9-83cf-ae6997e704d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:24.861935 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e4353-d0fb-43f9-83cf-ae6997e704d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:24.862214 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e4353-d0fb-43f9-83cf-ae6997e704d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:24.862488 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e4353-d0fb-43f9-83cf-ae6997e704d2 admin admin] PATH_INFO: `/v3/users/702a4174900c4701896578cd2892f336` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:24.863637 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3e4353-d0fb-43f9-83cf-ae6997e704d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=702a4174900c4701896578cd2892f336)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:24.865543 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3e4353-d0fb-43f9-83cf-ae6997e704d2 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:24.904746 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:24.911910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-7b3e4353-d0fb-43f9-83cf-ae6997e704d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '702a4174900c4701896578cd2892f336'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:24.919589 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-7b3e4353-d0fb-43f9-83cf-ae6997e704d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '702a4174900c4701896578cd2892f336'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:24.920378 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-7b3e4353-d0fb-43f9-83cf-ae6997e704d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '702a4174900c4701896578cd2892f336'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:24.921003 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-7b3e4353-d0fb-43f9-83cf-ae6997e704d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '702a4174900c4701896578cd2892f336'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:24.927795 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-7b3e4353-d0fb-43f9-83cf-ae6997e704d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '702a4174900c4701896578cd2892f336'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:24.945559 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1014/2027] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:14:24 2022] DELETE /identity/v3/users/702a4174900c4701896578cd2892f336 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:14:25.207746 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0555e-a503-484c-84e4-0edfadcdfb9d admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:25.209268 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-aba0555e-a503-484c-84e4-0edfadcdfb9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:25.222243 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0555e-a503-484c-84e4-0edfadcdfb9d admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:25.278783 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f035c7-9c95-4d1a-8100-cc022ac6b2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:25.279441 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f035c7-9c95-4d1a-8100-cc022ac6b2e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:25.279724 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f035c7-9c95-4d1a-8100-cc022ac6b2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:25.279997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f035c7-9c95-4d1a-8100-cc022ac6b2e1 admin admin] PATH_INFO: `/v3/projects/d07dc79346cb492fa539154818fc36eb` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:25.281251 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f035c7-9c95-4d1a-8100-cc022ac6b2e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d07dc79346cb492fa539154818fc36eb)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:25.283213 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f035c7-9c95-4d1a-8100-cc022ac6b2e1 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:25.293527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:25.330087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-09f035c7-9c95-4d1a-8100-cc022ac6b2e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd07dc79346cb492fa539154818fc36eb', 'request_id': 'req-09f035c7-9c95-4d1a-8100-cc022ac6b2e1'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:25.336800 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-09f035c7-9c95-4d1a-8100-cc022ac6b2e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd07dc79346cb492fa539154818fc36eb', 'request_id': 'req-09f035c7-9c95-4d1a-8100-cc022ac6b2e1'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:25.342525 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-09f035c7-9c95-4d1a-8100-cc022ac6b2e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd07dc79346cb492fa539154818fc36eb', 'request_id': 'req-09f035c7-9c95-4d1a-8100-cc022ac6b2e1'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:25.353442 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-09f035c7-9c95-4d1a-8100-cc022ac6b2e1 admin admin] The token cache is being invalidate because project d07dc79346cb492fa539154818fc36eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:14:25.353726 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-09f035c7-9c95-4d1a-8100-cc022ac6b2e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:25.354345 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-09f035c7-9c95-4d1a-8100-cc022ac6b2e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:25.355829 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1014/2028] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:14:25 2022] DELETE /identity/v3/projects/d07dc79346cb492fa539154818fc36eb => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:25.361147 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274eab0-ce84-46c9-bc67-0bbbe14b928d None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:25.361606 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274eab0-ce84-46c9-bc67-0bbbe14b928d None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:25.361753 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274eab0-ce84-46c9-bc67-0bbbe14b928d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:25.381581 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-1274eab0-ce84-46c9-bc67-0bbbe14b928d None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:25.382700 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-1274eab0-ce84-46c9-bc67-0bbbe14b928d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:25.439618 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1015/2029] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:25.447652 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a22cd-5597-4017-b7a9-250b9ecc9dd5 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:25.447935 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a22cd-5597-4017-b7a9-250b9ecc9dd5 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:25.448208 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a22cd-5597-4017-b7a9-250b9ecc9dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:25.470959 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-fb9a22cd-5597-4017-b7a9-250b9ecc9dd5 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:25.472169 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-fb9a22cd-5597-4017-b7a9-250b9ecc9dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:25.480645 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1015/2030] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:25.485145 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1274eab0-ce84-46c9-bc67-0bbbe14b928d None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:25.495046 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1274eab0-ce84-46c9-bc67-0bbbe14b928d None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:25.500238 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc04699-1249-46b8-b873-bd82ed058614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:25.500859 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc04699-1249-46b8-b873-bd82ed058614 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:25.501130 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc04699-1249-46b8-b873-bd82ed058614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:25.501397 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc04699-1249-46b8-b873-bd82ed058614 admin admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:25.502034 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc04699-1249-46b8-b873-bd82ed058614 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:14:25.502331 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc04699-1249-46b8-b873-bd82ed058614 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:25.503950 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc04699-1249-46b8-b873-bd82ed058614 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:25.515288 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1016/2031] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:14:25.519425 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a22cd-5597-4017-b7a9-250b9ecc9dd5 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:25.528327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a22cd-5597-4017-b7a9-250b9ecc9dd5 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:25.533017 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81e58b-54e0-40dd-b5ca-9bf57a938704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:25.533606 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81e58b-54e0-40dd-b5ca-9bf57a938704 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:25.533878 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81e58b-54e0-40dd-b5ca-9bf57a938704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:25.534142 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81e58b-54e0-40dd-b5ca-9bf57a938704 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:25.534765 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d81e58b-54e0-40dd-b5ca-9bf57a938704 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:25.536628 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d81e58b-54e0-40dd-b5ca-9bf57a938704 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:25.565482 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1016/2032] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:14:25 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:14:25.569591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc04699-1249-46b8-b873-bd82ed058614 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:25.579499 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc04699-1249-46b8-b873-bd82ed058614 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:25.635297 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa09df05-625f-4f3e-b865-6f0994e943d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:25.635830 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09df05-625f-4f3e-b865-6f0994e943d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:25.636106 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09df05-625f-4f3e-b865-6f0994e943d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:25.636376 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09df05-625f-4f3e-b865-6f0994e943d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:25.637003 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa09df05-625f-4f3e-b865-6f0994e943d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:25.638943 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa09df05-625f-4f3e-b865-6f0994e943d8 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:25.660591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1017/2033] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:14:25 2022] POST /identity/v3/users => generated 285 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:14:25.664681 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81e58b-54e0-40dd-b5ca-9bf57a938704 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:25.675483 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81e58b-54e0-40dd-b5ca-9bf57a938704 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:25.680016 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204a70b-9d01-41f4-9e54-c93e6d059667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:25.680612 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204a70b-9d01-41f4-9e54-c93e6d059667 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:25.680883 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204a70b-9d01-41f4-9e54-c93e6d059667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:25.681148 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204a70b-9d01-41f4-9e54-c93e6d059667 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:25.681813 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6204a70b-9d01-41f4-9e54-c93e6d059667 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:25.683405 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6204a70b-9d01-41f4-9e54-c93e6d059667 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:25.693795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1017/2034] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:14:25.697529 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa09df05-625f-4f3e-b865-6f0994e943d8 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:25.708873 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa09df05-625f-4f3e-b865-6f0994e943d8 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:25.714013 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b9a17d-83f3-4067-9d70-8f03f2bc9e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:25.714612 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9a17d-83f3-4067-9d70-8f03f2bc9e17 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:25.715288 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9a17d-83f3-4067-9d70-8f03f2bc9e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:25.715559 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9a17d-83f3-4067-9d70-8f03f2bc9e17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:25.716197 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b9a17d-83f3-4067-9d70-8f03f2bc9e17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:25.717770 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b9a17d-83f3-4067-9d70-8f03f2bc9e17 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:25.728725 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1018/2035] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:14:25.732818 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204a70b-9d01-41f4-9e54-c93e6d059667 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:25.743301 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204a70b-9d01-41f4-9e54-c93e6d059667 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:25.747876 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0c7d5-8ffc-47ed-b9ec-23a279f09b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:25.748567 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0c7d5-8ffc-47ed-b9ec-23a279f09b3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:25.748829 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0c7d5-8ffc-47ed-b9ec-23a279f09b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:25.749125 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0c7d5-8ffc-47ed-b9ec-23a279f09b3b admin admin] PATH_INFO: `/v3/projects/2c7c5afced6545c0b45bb7842169d9ba/users/740332fdeffc4389bb92478ff737bcd2/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:25.766851 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e0c7d5-8ffc-47ed-b9ec-23a279f09b3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c7c5afced6545c0b45bb7842169d9ba, user_id=740332fdeffc4389bb92478ff737bcd2, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:25.769496 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e0c7d5-8ffc-47ed-b9ec-23a279f09b3b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:25.784059 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1018/2036] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:14:25 2022] PUT /identity/v3/projects/2c7c5afced6545c0b45bb7842169d9ba/users/740332fdeffc4389bb92478ff737bcd2/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:25.795043 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ad036-980e-4211-9eb2-ab2ea16018d2 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:25.795325 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ad036-980e-4211-9eb2-ab2ea16018d2 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:25.795593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ad036-980e-4211-9eb2-ab2ea16018d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:25.836552 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-dc9ad036-980e-4211-9eb2-ab2ea16018d2 None None] MFA Rules not processed for user `740332fdeffc4389bb92478ff737bcd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:25.838376 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-dc9ad036-980e-4211-9eb2-ab2ea16018d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:25.873413 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:25.917388 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1019/2037] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:25.930216 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0c7d5-8ffc-47ed-b9ec-23a279f09b3b admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:25.931993 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d7e0c7d5-8ffc-47ed-b9ec-23a279f09b3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:25.992205 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0c7d5-8ffc-47ed-b9ec-23a279f09b3b admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:25.995766 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2ce199-f92d-4ffe-b697-c10da13fdaac service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2092a9639067461c9a837a891f56889b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:25.996367 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ce199-f92d-4ffe-b697-c10da13fdaac service glance] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:25.996640 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ce199-f92d-4ffe-b697-c10da13fdaac service glance] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:25.996907 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ce199-f92d-4ffe-b697-c10da13fdaac service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:26.005019 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2ce199-f92d-4ffe-b697-c10da13fdaac service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:26.006408 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2ce199-f92d-4ffe-b697-c10da13fdaac service glance] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:26.015115 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1019/2038] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:14:25 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:14:32.675819 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ad036-980e-4211-9eb2-ab2ea16018d2 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:32.677247 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-dc9ad036-980e-4211-9eb2-ab2ea16018d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:32.748438 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ad036-980e-4211-9eb2-ab2ea16018d2 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:32.757041 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126248a2-b75a-4383-ac2d-cd5230db33a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:32.757741 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126248a2-b75a-4383-ac2d-cd5230db33a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:32.758023 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126248a2-b75a-4383-ac2d-cd5230db33a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:32.758290 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126248a2-b75a-4383-ac2d-cd5230db33a3 admin admin] PATH_INFO: `/v3/users/6abc2245d0dc494da8332ab0fcfca4c6` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:32.759490 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126248a2-b75a-4383-ac2d-cd5230db33a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6abc2245d0dc494da8332ab0fcfca4c6)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:32.761919 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126248a2-b75a-4383-ac2d-cd5230db33a3 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:32.836574 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-126248a2-b75a-4383-ac2d-cd5230db33a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6abc2245d0dc494da8332ab0fcfca4c6'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:32.847676 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-126248a2-b75a-4383-ac2d-cd5230db33a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6abc2245d0dc494da8332ab0fcfca4c6'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:32.848354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-126248a2-b75a-4383-ac2d-cd5230db33a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6abc2245d0dc494da8332ab0fcfca4c6'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:32.848915 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-126248a2-b75a-4383-ac2d-cd5230db33a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6abc2245d0dc494da8332ab0fcfca4c6'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:32.856563 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-126248a2-b75a-4383-ac2d-cd5230db33a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6abc2245d0dc494da8332ab0fcfca4c6'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:32.875057 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1020/2039] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:14:32 2022] DELETE /identity/v3/users/6abc2245d0dc494da8332ab0fcfca4c6 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:32.879614 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913440cf-b085-4541-b747-ba69a900cb85 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:32.880110 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913440cf-b085-4541-b747-ba69a900cb85 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:32.880260 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913440cf-b085-4541-b747-ba69a900cb85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:32.902666 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-913440cf-b085-4541-b747-ba69a900cb85 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:32.904129 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-913440cf-b085-4541-b747-ba69a900cb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:32.945393 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:32.983833 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1020/2040] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:32.994631 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126248a2-b75a-4383-ac2d-cd5230db33a3 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:32.996359 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-126248a2-b75a-4383-ac2d-cd5230db33a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:33.055840 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:33.088272 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126248a2-b75a-4383-ac2d-cd5230db33a3 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:33.099440 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20416fbd-b592-45db-93f1-71eba2e459d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:33.100110 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20416fbd-b592-45db-93f1-71eba2e459d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:33.100295 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20416fbd-b592-45db-93f1-71eba2e459d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:33.100581 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20416fbd-b592-45db-93f1-71eba2e459d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:33.111340 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20416fbd-b592-45db-93f1-71eba2e459d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:33.112781 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20416fbd-b592-45db-93f1-71eba2e459d1 service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:33.122622 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1021/2041] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:14:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:14:33.645330 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913440cf-b085-4541-b747-ba69a900cb85 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:33.647122 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-913440cf-b085-4541-b747-ba69a900cb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:33.663270 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913440cf-b085-4541-b747-ba69a900cb85 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:33.669829 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db49393-3e5e-4f0a-8bb8-c3b1a5454451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:33.671425 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db49393-3e5e-4f0a-8bb8-c3b1a5454451 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:33.671731 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db49393-3e5e-4f0a-8bb8-c3b1a5454451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:33.672009 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db49393-3e5e-4f0a-8bb8-c3b1a5454451 admin admin] PATH_INFO: `/v3/projects/bac4a3d979ea4877a5cc6ce3ca947e3b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:33.673265 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db49393-3e5e-4f0a-8bb8-c3b1a5454451 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bac4a3d979ea4877a5cc6ce3ca947e3b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:33.676382 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db49393-3e5e-4f0a-8bb8-c3b1a5454451 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:33.747674 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-4db49393-3e5e-4f0a-8bb8-c3b1a5454451 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bac4a3d979ea4877a5cc6ce3ca947e3b', 'request_id': 'req-4db49393-3e5e-4f0a-8bb8-c3b1a5454451'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:33.758441 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-4db49393-3e5e-4f0a-8bb8-c3b1a5454451 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bac4a3d979ea4877a5cc6ce3ca947e3b', 'request_id': 'req-4db49393-3e5e-4f0a-8bb8-c3b1a5454451'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:33.766238 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-4db49393-3e5e-4f0a-8bb8-c3b1a5454451 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bac4a3d979ea4877a5cc6ce3ca947e3b', 'request_id': 'req-4db49393-3e5e-4f0a-8bb8-c3b1a5454451'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:33.780852 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-4db49393-3e5e-4f0a-8bb8-c3b1a5454451 admin admin] The token cache is being invalidate because project bac4a3d979ea4877a5cc6ce3ca947e3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:14:33.781191 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-4db49393-3e5e-4f0a-8bb8-c3b1a5454451 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:33.789232 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-4db49393-3e5e-4f0a-8bb8-c3b1a5454451 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:33.791278 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1021/2042] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:14:33 2022] DELETE /identity/v3/projects/bac4a3d979ea4877a5cc6ce3ca947e3b => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:33.796668 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20416fbd-b592-45db-93f1-71eba2e459d1 service neutron] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:33.823154 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20416fbd-b592-45db-93f1-71eba2e459d1 service neutron] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:33.918620 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c89758-aa4f-45cf-b2ac-4cce49fa5f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:33.919768 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c89758-aa4f-45cf-b2ac-4cce49fa5f35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:33.920248 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c89758-aa4f-45cf-b2ac-4cce49fa5f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:33.921417 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c89758-aa4f-45cf-b2ac-4cce49fa5f35 admin admin] PATH_INFO: `/v3/users/8886ddd9b77244578be5d25cad17fb95` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:33.923056 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c89758-aa4f-45cf-b2ac-4cce49fa5f35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8886ddd9b77244578be5d25cad17fb95)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:33.925317 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c89758-aa4f-45cf-b2ac-4cce49fa5f35 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:33.996748 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-07c89758-aa4f-45cf-b2ac-4cce49fa5f35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8886ddd9b77244578be5d25cad17fb95'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:34.007652 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-07c89758-aa4f-45cf-b2ac-4cce49fa5f35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8886ddd9b77244578be5d25cad17fb95'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:34.008418 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-07c89758-aa4f-45cf-b2ac-4cce49fa5f35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8886ddd9b77244578be5d25cad17fb95'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:34.009129 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-07c89758-aa4f-45cf-b2ac-4cce49fa5f35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8886ddd9b77244578be5d25cad17fb95'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:34.017364 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-07c89758-aa4f-45cf-b2ac-4cce49fa5f35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8886ddd9b77244578be5d25cad17fb95'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:34.038040 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1022/2043] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:14:33 2022] DELETE /identity/v3/users/8886ddd9b77244578be5d25cad17fb95 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:34.743921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db49393-3e5e-4f0a-8bb8-c3b1a5454451 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:34.746465 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-4db49393-3e5e-4f0a-8bb8-c3b1a5454451 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:34.761818 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db49393-3e5e-4f0a-8bb8-c3b1a5454451 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:34.861488 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356fb08-0012-4394-a455-7a4b2e426572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:34.862535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356fb08-0012-4394-a455-7a4b2e426572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:34.862912 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356fb08-0012-4394-a455-7a4b2e426572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:34.863186 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356fb08-0012-4394-a455-7a4b2e426572 admin admin] PATH_INFO: `/v3/projects/0aef64154651444f9962020581d958db` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:34.864424 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c356fb08-0012-4394-a455-7a4b2e426572 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0aef64154651444f9962020581d958db)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:34.866321 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c356fb08-0012-4394-a455-7a4b2e426572 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:34.938010 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c356fb08-0012-4394-a455-7a4b2e426572 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0aef64154651444f9962020581d958db', 'request_id': 'req-c356fb08-0012-4394-a455-7a4b2e426572'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:34.950391 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c356fb08-0012-4394-a455-7a4b2e426572 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0aef64154651444f9962020581d958db', 'request_id': 'req-c356fb08-0012-4394-a455-7a4b2e426572'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:34.963080 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:34.965483 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c356fb08-0012-4394-a455-7a4b2e426572 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0aef64154651444f9962020581d958db', 'request_id': 'req-c356fb08-0012-4394-a455-7a4b2e426572'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:34.981453 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c356fb08-0012-4394-a455-7a4b2e426572 admin admin] The token cache is being invalidate because project 0aef64154651444f9962020581d958db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:14:34.982052 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c356fb08-0012-4394-a455-7a4b2e426572 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:34.983200 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c356fb08-0012-4394-a455-7a4b2e426572 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:34.985273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1022/2044] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:14:34 2022] DELETE /identity/v3/projects/0aef64154651444f9962020581d958db => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:34.990550 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c89758-aa4f-45cf-b2ac-4cce49fa5f35 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:34.991562 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-07c89758-aa4f-45cf-b2ac-4cce49fa5f35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:35.010675 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c89758-aa4f-45cf-b2ac-4cce49fa5f35 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:35.083246 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:35.096149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f09195-c20d-48d6-a258-d551c2a0ed2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:35.098529 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f09195-c20d-48d6-a258-d551c2a0ed2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:35.099047 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f09195-c20d-48d6-a258-d551c2a0ed2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:35.100228 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f09195-c20d-48d6-a258-d551c2a0ed2f admin admin] PATH_INFO: `/v3/users/3609dd077d69452dae8cbc1bd642d78b` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:35.101878 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f09195-c20d-48d6-a258-d551c2a0ed2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3609dd077d69452dae8cbc1bd642d78b)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:35.104675 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f09195-c20d-48d6-a258-d551c2a0ed2f admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:35.176444 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-c3f09195-c20d-48d6-a258-d551c2a0ed2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3609dd077d69452dae8cbc1bd642d78b'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:35.199740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-c3f09195-c20d-48d6-a258-d551c2a0ed2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3609dd077d69452dae8cbc1bd642d78b'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:35.199740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-c3f09195-c20d-48d6-a258-d551c2a0ed2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3609dd077d69452dae8cbc1bd642d78b'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:35.199740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-c3f09195-c20d-48d6-a258-d551c2a0ed2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3609dd077d69452dae8cbc1bd642d78b'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:35.204075 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-c3f09195-c20d-48d6-a258-d551c2a0ed2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3609dd077d69452dae8cbc1bd642d78b'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:35.226582 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1023/2045] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:14:34 2022] DELETE /identity/v3/users/3609dd077d69452dae8cbc1bd642d78b => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:35.546975 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356fb08-0012-4394-a455-7a4b2e426572 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:35.548583 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c356fb08-0012-4394-a455-7a4b2e426572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:35.565530 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356fb08-0012-4394-a455-7a4b2e426572 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:35.633258 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c0fc5f-7fca-42b7-ab77-08ff7240ae26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:35.633902 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c0fc5f-7fca-42b7-ab77-08ff7240ae26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:35.634182 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c0fc5f-7fca-42b7-ab77-08ff7240ae26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:35.634453 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c0fc5f-7fca-42b7-ab77-08ff7240ae26 admin admin] PATH_INFO: `/v3/projects/024a41a21a3449ffb2c4301b287686bb` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:35.635608 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c0fc5f-7fca-42b7-ab77-08ff7240ae26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=024a41a21a3449ffb2c4301b287686bb)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:35.637530 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c0fc5f-7fca-42b7-ab77-08ff7240ae26 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:35.694402 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-07c0fc5f-7fca-42b7-ab77-08ff7240ae26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '024a41a21a3449ffb2c4301b287686bb', 'request_id': 'req-07c0fc5f-7fca-42b7-ab77-08ff7240ae26'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:35.705254 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-07c0fc5f-7fca-42b7-ab77-08ff7240ae26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '024a41a21a3449ffb2c4301b287686bb', 'request_id': 'req-07c0fc5f-7fca-42b7-ab77-08ff7240ae26'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:35.713645 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-07c0fc5f-7fca-42b7-ab77-08ff7240ae26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '024a41a21a3449ffb2c4301b287686bb', 'request_id': 'req-07c0fc5f-7fca-42b7-ab77-08ff7240ae26'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:35.727386 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-07c0fc5f-7fca-42b7-ab77-08ff7240ae26 admin admin] The token cache is being invalidate because project 024a41a21a3449ffb2c4301b287686bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:14:35.727792 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-07c0fc5f-7fca-42b7-ab77-08ff7240ae26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:35.728568 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-07c0fc5f-7fca-42b7-ab77-08ff7240ae26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:35.735016 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1023/2046] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:14:35 2022] DELETE /identity/v3/projects/024a41a21a3449ffb2c4301b287686bb => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:35.737409 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a53bed-5edc-4168-a58f-a96e1babf290 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:35.737722 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a53bed-5edc-4168-a58f-a96e1babf290 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:35.738321 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a53bed-5edc-4168-a58f-a96e1babf290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:35.763239 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-75a53bed-5edc-4168-a58f-a96e1babf290 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:35.764514 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-75a53bed-5edc-4168-a58f-a96e1babf290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:35.841292 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1024/2047] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:35.849503 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086bd9b-14dc-463c-93d0-8df1fef288ca None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:35.849992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086bd9b-14dc-463c-93d0-8df1fef288ca None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:35.850168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086bd9b-14dc-463c-93d0-8df1fef288ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:35.875176 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-9086bd9b-14dc-463c-93d0-8df1fef288ca None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:35.876485 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-9086bd9b-14dc-463c-93d0-8df1fef288ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:35.884660 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1024/2048] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:14:35.889150 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a53bed-5edc-4168-a58f-a96e1babf290 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:35.899521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a53bed-5edc-4168-a58f-a96e1babf290 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:35.904158 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86288bb2-2abd-420f-8d77-8951f4734959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:35.904894 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86288bb2-2abd-420f-8d77-8951f4734959 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:35.905885 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86288bb2-2abd-420f-8d77-8951f4734959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:35.906217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86288bb2-2abd-420f-8d77-8951f4734959 admin admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:35.906999 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86288bb2-2abd-420f-8d77-8951f4734959 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:14:35.907391 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86288bb2-2abd-420f-8d77-8951f4734959 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:35.909052 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86288bb2-2abd-420f-8d77-8951f4734959 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:35.922866 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1025/2049] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:14:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:14:35.926571 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086bd9b-14dc-463c-93d0-8df1fef288ca None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:35.935908 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086bd9b-14dc-463c-93d0-8df1fef288ca None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:35.940799 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3acbe0f-da60-43e9-92fa-75b78d8509b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:35.941492 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3acbe0f-da60-43e9-92fa-75b78d8509b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:35.941772 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3acbe0f-da60-43e9-92fa-75b78d8509b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:35.942039 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3acbe0f-da60-43e9-92fa-75b78d8509b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:35.942659 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3acbe0f-da60-43e9-92fa-75b78d8509b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:35.944570 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3acbe0f-da60-43e9-92fa-75b78d8509b0 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:35.979801 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1025/2050] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:14:35 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:14:35.980940 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86288bb2-2abd-420f-8d77-8951f4734959 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:35.991379 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86288bb2-2abd-420f-8d77-8951f4734959 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:36.067286 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc7619a-6930-447b-8ab9-c3e10ed0ca25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:36.067812 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7619a-6930-447b-8ab9-c3e10ed0ca25 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:36.068086 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7619a-6930-447b-8ab9-c3e10ed0ca25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:36.068352 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7619a-6930-447b-8ab9-c3e10ed0ca25 admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:36.068996 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc7619a-6930-447b-8ab9-c3e10ed0ca25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:36.070955 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc7619a-6930-447b-8ab9-c3e10ed0ca25 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:36.083483 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:36.109228 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1026/2051] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:14:35 2022] POST /identity/v3/users => generated 287 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:14:36.110710 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3acbe0f-da60-43e9-92fa-75b78d8509b0 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:36.130252 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3acbe0f-da60-43e9-92fa-75b78d8509b0 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:36.135905 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42b7dc-aa95-4ab5-9c0c-f3a09833bc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:36.136526 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42b7dc-aa95-4ab5-9c0c-f3a09833bc25 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:36.136801 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42b7dc-aa95-4ab5-9c0c-f3a09833bc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:36.137068 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42b7dc-aa95-4ab5-9c0c-f3a09833bc25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:36.137932 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42b7dc-aa95-4ab5-9c0c-f3a09833bc25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:36.140325 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42b7dc-aa95-4ab5-9c0c-f3a09833bc25 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:36.158699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1026/2052] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:14:36.160392 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc7619a-6930-447b-8ab9-c3e10ed0ca25 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:36.172155 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc7619a-6930-447b-8ab9-c3e10ed0ca25 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:36.183069 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba16035-8001-4cd7-8c14-e77b9187a037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:36.183842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba16035-8001-4cd7-8c14-e77b9187a037 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:36.184240 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba16035-8001-4cd7-8c14-e77b9187a037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:36.184620 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba16035-8001-4cd7-8c14-e77b9187a037 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:36.185419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba16035-8001-4cd7-8c14-e77b9187a037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:36.188270 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba16035-8001-4cd7-8c14-e77b9187a037 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:36.205289 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1027/2053] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:14:36.209871 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42b7dc-aa95-4ab5-9c0c-f3a09833bc25 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:36.220646 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42b7dc-aa95-4ab5-9c0c-f3a09833bc25 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:36.231078 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41dd109-3c29-4d8c-9fe2-8371c72e1e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:36.231903 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41dd109-3c29-4d8c-9fe2-8371c72e1e74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:36.232263 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41dd109-3c29-4d8c-9fe2-8371c72e1e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:36.232629 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41dd109-3c29-4d8c-9fe2-8371c72e1e74 admin admin] PATH_INFO: `/v3/projects/7f946cd56d394d75935e5277811c39c7/users/c7400d293884411b94618c5f3aac8c43/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:36.251550 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41dd109-3c29-4d8c-9fe2-8371c72e1e74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f946cd56d394d75935e5277811c39c7, user_id=c7400d293884411b94618c5f3aac8c43, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:36.254636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41dd109-3c29-4d8c-9fe2-8371c72e1e74 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:36.271256 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1027/2054] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:14:36 2022] PUT /identity/v3/projects/7f946cd56d394d75935e5277811c39c7/users/c7400d293884411b94618c5f3aac8c43/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:36.283788 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8720f28-ee16-4c0b-9ca6-1eab94671ed0 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:36.283788 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8720f28-ee16-4c0b-9ca6-1eab94671ed0 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:36.284099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8720f28-ee16-4c0b-9ca6-1eab94671ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:36.343543 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-b8720f28-ee16-4c0b-9ca6-1eab94671ed0 None None] MFA Rules not processed for user `c7400d293884411b94618c5f3aac8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:36.344783 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-b8720f28-ee16-4c0b-9ca6-1eab94671ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:36.459833 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1028/2055] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:36.472925 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41dd109-3c29-4d8c-9fe2-8371c72e1e74 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:36.474769 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e41dd109-3c29-4d8c-9fe2-8371c72e1e74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:36.505230 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:36.555910 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41dd109-3c29-4d8c-9fe2-8371c72e1e74 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:36.560030 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc4f09-b120-46ae-a59e-0234cf7d1a15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:36.560792 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc4f09-b120-46ae-a59e-0234cf7d1a15 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:36.561139 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc4f09-b120-46ae-a59e-0234cf7d1a15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:36.562010 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc4f09-b120-46ae-a59e-0234cf7d1a15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:36.572982 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc4f09-b120-46ae-a59e-0234cf7d1a15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:36.574414 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc4f09-b120-46ae-a59e-0234cf7d1a15 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:36.585174 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1028/2056] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:14:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:14:36.961105 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a64710-69bf-4138-a521-8ddc727a89be None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:36.963215 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a64710-69bf-4138-a521-8ddc727a89be None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:36.963898 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a64710-69bf-4138-a521-8ddc727a89be None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:36.966170 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1029/2057] 10.222.0.42 () {54 vars in 921 bytes} [Mon Sep 12 14:14:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:14:36.979576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96a8ce-a150-48f9-bcdd-a8c20d3e6a23 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:36.980113 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96a8ce-a150-48f9-bcdd-a8c20d3e6a23 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:36.980113 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96a8ce-a150-48f9-bcdd-a8c20d3e6a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:37.003118 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-cc96a8ce-a150-48f9-bcdd-a8c20d3e6a23 None None] MFA Rules not processed for user `ce508bb9aa164a7fb1831a9b5ff5b932`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:37.004462 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-cc96a8ce-a150-48f9-bcdd-a8c20d3e6a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:37.085021 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a64710-69bf-4138-a521-8ddc727a89be None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:37.086900 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-c5a64710-69bf-4138-a521-8ddc727a89be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:37.110751 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1029/2058] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Sep 12 14:14:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:37.122276 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc96a8ce-a150-48f9-bcdd-a8c20d3e6a23 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:37.123744 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-cc96a8ce-a150-48f9-bcdd-a8c20d3e6a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:37.137309 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc96a8ce-a150-48f9-bcdd-a8c20d3e6a23 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:37.142616 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3904fe6-302e-4600-a4c4-f845b279669f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce508bb9aa164a7fb1831a9b5ff5b932', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:37.143255 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3904fe6-302e-4600-a4c4-f845b279669f service placement] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:37.143533 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3904fe6-302e-4600-a4c4-f845b279669f service placement] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:37.143807 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3904fe6-302e-4600-a4c4-f845b279669f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:37.152638 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3904fe6-302e-4600-a4c4-f845b279669f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:37.154591 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3904fe6-302e-4600-a4c4-f845b279669f service placement] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:37.165123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1030/2059] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:14:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:14:37.174729 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a64710-69bf-4138-a521-8ddc727a89be None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:37.179206 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60462830-a028-4a47-b0b5-2d39dd3a754e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:37.179830 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60462830-a028-4a47-b0b5-2d39dd3a754e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:37.180102 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60462830-a028-4a47-b0b5-2d39dd3a754e admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:37.180372 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60462830-a028-4a47-b0b5-2d39dd3a754e admin admin] PATH_INFO: `/v3/users/1c6cd05ddb3d4d14b2a18bc45401e73d` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:37.181503 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60462830-a028-4a47-b0b5-2d39dd3a754e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c6cd05ddb3d4d14b2a18bc45401e73d)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:37.183816 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60462830-a028-4a47-b0b5-2d39dd3a754e admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:37.230150 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-60462830-a028-4a47-b0b5-2d39dd3a754e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c6cd05ddb3d4d14b2a18bc45401e73d'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:37.237865 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-60462830-a028-4a47-b0b5-2d39dd3a754e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c6cd05ddb3d4d14b2a18bc45401e73d'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:37.238496 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-60462830-a028-4a47-b0b5-2d39dd3a754e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c6cd05ddb3d4d14b2a18bc45401e73d'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:37.239160 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-60462830-a028-4a47-b0b5-2d39dd3a754e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c6cd05ddb3d4d14b2a18bc45401e73d'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:37.245946 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-60462830-a028-4a47-b0b5-2d39dd3a754e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c6cd05ddb3d4d14b2a18bc45401e73d'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:37.252814 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:INSERT=3,keystone:SELECT=88,keystone:DELETE=9 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:37.265754 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1030/2060] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:14:37 2022] DELETE /identity/v3/users/1c6cd05ddb3d4d14b2a18bc45401e73d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:14:37.436421 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3904fe6-302e-4600-a4c4-f845b279669f service placement] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:37.447662 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3904fe6-302e-4600-a4c4-f845b279669f service placement] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:37.504143 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b85750-a81b-4518-8c21-4eeeadc5c461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:37.504759 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b85750-a81b-4518-8c21-4eeeadc5c461 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:37.505035 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b85750-a81b-4518-8c21-4eeeadc5c461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:37.505305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b85750-a81b-4518-8c21-4eeeadc5c461 admin admin] PATH_INFO: `/v3/projects/e73505faee9c4bacbf2456e2d69c56f2` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:37.506394 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b85750-a81b-4518-8c21-4eeeadc5c461 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e73505faee9c4bacbf2456e2d69c56f2)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:37.508291 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b85750-a81b-4518-8c21-4eeeadc5c461 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:37.551026 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:37.554079 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b8b85750-a81b-4518-8c21-4eeeadc5c461 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e73505faee9c4bacbf2456e2d69c56f2', 'request_id': 'req-b8b85750-a81b-4518-8c21-4eeeadc5c461'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:37.561489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b8b85750-a81b-4518-8c21-4eeeadc5c461 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e73505faee9c4bacbf2456e2d69c56f2', 'request_id': 'req-b8b85750-a81b-4518-8c21-4eeeadc5c461'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:37.567329 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b8b85750-a81b-4518-8c21-4eeeadc5c461 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e73505faee9c4bacbf2456e2d69c56f2', 'request_id': 'req-b8b85750-a81b-4518-8c21-4eeeadc5c461'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:37.578672 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b8b85750-a81b-4518-8c21-4eeeadc5c461 admin admin] The token cache is being invalidate because project e73505faee9c4bacbf2456e2d69c56f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:14:37.578992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b8b85750-a81b-4518-8c21-4eeeadc5c461 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:37.579643 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b8b85750-a81b-4518-8c21-4eeeadc5c461 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:37.581167 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1031/2061] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:14:37 2022] DELETE /identity/v3/projects/e73505faee9c4bacbf2456e2d69c56f2 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:37.585947 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e545d9-e8ea-4c72-a754-0a68dcf72949 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:37.586319 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e545d9-e8ea-4c72-a754-0a68dcf72949 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:37.586662 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e545d9-e8ea-4c72-a754-0a68dcf72949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:37.606307 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-94e545d9-e8ea-4c72-a754-0a68dcf72949 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:37.607510 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-94e545d9-e8ea-4c72-a754-0a68dcf72949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:37.664239 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1031/2062] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:37.672199 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfd671-5aad-4457-846b-137c4e110aa8 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:37.672627 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfd671-5aad-4457-846b-137c4e110aa8 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:37.672908 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfd671-5aad-4457-846b-137c4e110aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:37.692846 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-37dfd671-5aad-4457-846b-137c4e110aa8 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:37.694039 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-37dfd671-5aad-4457-846b-137c4e110aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:37.702479 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1032/2063] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:14:37.706503 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e545d9-e8ea-4c72-a754-0a68dcf72949 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:37.715994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e545d9-e8ea-4c72-a754-0a68dcf72949 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:37.720481 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9aa86-b932-4caf-839c-54ec5463736f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:37.721067 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9aa86-b932-4caf-839c-54ec5463736f admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:37.721341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9aa86-b932-4caf-839c-54ec5463736f admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:37.721606 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9aa86-b932-4caf-839c-54ec5463736f admin admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:37.722271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9aa86-b932-4caf-839c-54ec5463736f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:14:37.722569 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9aa86-b932-4caf-839c-54ec5463736f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:37.724175 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9aa86-b932-4caf-839c-54ec5463736f admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:37.735386 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1032/2064] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:14:37.739526 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dfd671-5aad-4457-846b-137c4e110aa8 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:37.749235 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dfd671-5aad-4457-846b-137c4e110aa8 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:37.757688 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba32405-32a0-41fd-b70c-615708049dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:37.758284 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba32405-32a0-41fd-b70c-615708049dfa admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:37.758563 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba32405-32a0-41fd-b70c-615708049dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:37.758891 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba32405-32a0-41fd-b70c-615708049dfa admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:37.759522 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba32405-32a0-41fd-b70c-615708049dfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:37.761540 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba32405-32a0-41fd-b70c-615708049dfa admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:37.788936 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1033/2065] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:14:37 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:14:37.793167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9aa86-b932-4caf-839c-54ec5463736f admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:37.803160 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9aa86-b932-4caf-839c-54ec5463736f admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:37.863618 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3215197-66f0-4490-9264-7dd6d9f97c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:37.864230 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3215197-66f0-4490-9264-7dd6d9f97c9c admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:37.864508 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3215197-66f0-4490-9264-7dd6d9f97c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:37.864801 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3215197-66f0-4490-9264-7dd6d9f97c9c admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:37.865438 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3215197-66f0-4490-9264-7dd6d9f97c9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:37.867386 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3215197-66f0-4490-9264-7dd6d9f97c9c admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:37.888116 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1033/2066] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:14:37 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:14:37.892803 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba32405-32a0-41fd-b70c-615708049dfa admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:37.903110 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba32405-32a0-41fd-b70c-615708049dfa admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:37.907979 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f92f59-b04b-4ce6-996f-d214f5edbf6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:37.908607 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f92f59-b04b-4ce6-996f-d214f5edbf6e admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:37.908885 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f92f59-b04b-4ce6-996f-d214f5edbf6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:37.909157 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f92f59-b04b-4ce6-996f-d214f5edbf6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:37.909798 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f92f59-b04b-4ce6-996f-d214f5edbf6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:37.911477 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f92f59-b04b-4ce6-996f-d214f5edbf6e admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:37.921331 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1034/2067] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:14:37.925192 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3215197-66f0-4490-9264-7dd6d9f97c9c admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:37.935049 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3215197-66f0-4490-9264-7dd6d9f97c9c admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:37.940605 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f212a95-a2e2-46d0-98af-41bd0953f7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:37.941196 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f212a95-a2e2-46d0-98af-41bd0953f7e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:37.941477 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f212a95-a2e2-46d0-98af-41bd0953f7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:37.941749 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f212a95-a2e2-46d0-98af-41bd0953f7e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:37.942387 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f212a95-a2e2-46d0-98af-41bd0953f7e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:37.944001 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f212a95-a2e2-46d0-98af-41bd0953f7e1 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:37.958080 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1034/2068] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:14:37.959406 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f92f59-b04b-4ce6-996f-d214f5edbf6e admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:37.969721 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f92f59-b04b-4ce6-996f-d214f5edbf6e admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:37.974396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac47cee-8a06-4e6c-8bc2-9af5380794a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:37.975085 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac47cee-8a06-4e6c-8bc2-9af5380794a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:37.975392 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac47cee-8a06-4e6c-8bc2-9af5380794a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:37.975830 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac47cee-8a06-4e6c-8bc2-9af5380794a2 admin admin] PATH_INFO: `/v3/projects/06a6d09a62e84a01a95c65a743fc91a0/users/6437da5191e5497babe94df114c09f3f/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:37.990428 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac47cee-8a06-4e6c-8bc2-9af5380794a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06a6d09a62e84a01a95c65a743fc91a0, user_id=6437da5191e5497babe94df114c09f3f, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:37.993079 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac47cee-8a06-4e6c-8bc2-9af5380794a2 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:38.007791 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1035/2069] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:14:37 2022] PUT /identity/v3/projects/06a6d09a62e84a01a95c65a743fc91a0/users/6437da5191e5497babe94df114c09f3f/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:14:38.018541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd74ea4-7ea7-4f41-b48c-a45efb777e34 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:38.018820 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd74ea4-7ea7-4f41-b48c-a45efb777e34 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:38.019161 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd74ea4-7ea7-4f41-b48c-a45efb777e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:38.048025 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-edd74ea4-7ea7-4f41-b48c-a45efb777e34 None None] MFA Rules not processed for user `6437da5191e5497babe94df114c09f3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:38.049222 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-edd74ea4-7ea7-4f41-b48c-a45efb777e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:38.096915 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:38.121653 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1035/2070] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:14:38.126635 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac47cee-8a06-4e6c-8bc2-9af5380794a2 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:38.198807 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac47cee-8a06-4e6c-8bc2-9af5380794a2 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:38.203532 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09e39b-137e-4a55-93cf-ff636d295d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:38.204137 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09e39b-137e-4a55-93cf-ff636d295d8c admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:38.204421 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09e39b-137e-4a55-93cf-ff636d295d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:38.204699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09e39b-137e-4a55-93cf-ff636d295d8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:38.205329 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e09e39b-137e-4a55-93cf-ff636d295d8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:38.207244 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e09e39b-137e-4a55-93cf-ff636d295d8c admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:38.234832 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1036/2071] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:14:38 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:14:38.238897 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd74ea4-7ea7-4f41-b48c-a45efb777e34 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:38.249948 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd74ea4-7ea7-4f41-b48c-a45efb777e34 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:38.307791 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000e48d2-9bde-43dc-982c-8d74046c1efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:38.308384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e48d2-9bde-43dc-982c-8d74046c1efb admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:38.308660 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e48d2-9bde-43dc-982c-8d74046c1efb admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:38.308932 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e48d2-9bde-43dc-982c-8d74046c1efb admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:38.309556 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000e48d2-9bde-43dc-982c-8d74046c1efb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:38.312095 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000e48d2-9bde-43dc-982c-8d74046c1efb admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:38.333238 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1036/2072] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:14:38 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:14:38.337188 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09e39b-137e-4a55-93cf-ff636d295d8c admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:38.347546 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09e39b-137e-4a55-93cf-ff636d295d8c admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:38.352131 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0f970-8385-4b69-9df8-4508c5486d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:38.352721 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0f970-8385-4b69-9df8-4508c5486d88 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:38.352998 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0f970-8385-4b69-9df8-4508c5486d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:38.353270 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0f970-8385-4b69-9df8-4508c5486d88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:38.353914 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a0f970-8385-4b69-9df8-4508c5486d88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:38.355728 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a0f970-8385-4b69-9df8-4508c5486d88 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:38.367077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1037/2073] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:14:38.372331 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000e48d2-9bde-43dc-982c-8d74046c1efb admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:38.383212 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000e48d2-9bde-43dc-982c-8d74046c1efb admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:38.387843 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca82b3-d67b-4dcd-bf32-93aa5dd91c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:38.388481 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca82b3-d67b-4dcd-bf32-93aa5dd91c87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:38.388756 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca82b3-d67b-4dcd-bf32-93aa5dd91c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:38.389021 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca82b3-d67b-4dcd-bf32-93aa5dd91c87 admin admin] PATH_INFO: `/v3/projects/a3ef98e32a0941948b911bea165a4ecc/users/0ae8addce37444f3a629aa059a3efd9a/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:38.403632 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfca82b3-d67b-4dcd-bf32-93aa5dd91c87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3ef98e32a0941948b911bea165a4ecc, user_id=0ae8addce37444f3a629aa059a3efd9a, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:38.406763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfca82b3-d67b-4dcd-bf32-93aa5dd91c87 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:38.419314 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1037/2074] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:14:38 2022] PUT /identity/v3/projects/a3ef98e32a0941948b911bea165a4ecc/users/0ae8addce37444f3a629aa059a3efd9a/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:38.423564 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0f970-8385-4b69-9df8-4508c5486d88 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:38.433509 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0f970-8385-4b69-9df8-4508c5486d88 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:38.476376 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:38.493821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded57af-af1f-4c73-9e73-973a57bb599f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:38.494430 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded57af-af1f-4c73-9e73-973a57bb599f admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:38.494710 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded57af-af1f-4c73-9e73-973a57bb599f admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:38.495016 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded57af-af1f-4c73-9e73-973a57bb599f admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:38.495656 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded57af-af1f-4c73-9e73-973a57bb599f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:38.497218 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded57af-af1f-4c73-9e73-973a57bb599f admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:38.506939 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1038/2075] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:14:38.511178 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca82b3-d67b-4dcd-bf32-93aa5dd91c87 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:38.521306 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca82b3-d67b-4dcd-bf32-93aa5dd91c87 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:38.525975 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bad4b4-899d-477d-8ec5-51e521fbb222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:38.526605 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bad4b4-899d-477d-8ec5-51e521fbb222 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:38.527013 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bad4b4-899d-477d-8ec5-51e521fbb222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:38.527283 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bad4b4-899d-477d-8ec5-51e521fbb222 admin admin] PATH_INFO: `/v3/projects/a3ef98e32a0941948b911bea165a4ecc/users/0ae8addce37444f3a629aa059a3efd9a/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:38.529159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bad4b4-899d-477d-8ec5-51e521fbb222 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3ef98e32a0941948b911bea165a4ecc, user_id=0ae8addce37444f3a629aa059a3efd9a, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:38.532291 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bad4b4-899d-477d-8ec5-51e521fbb222 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:38.542136 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1038/2076] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:14:38 2022] PUT /identity/v3/projects/a3ef98e32a0941948b911bea165a4ecc/users/0ae8addce37444f3a629aa059a3efd9a/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:38.556597 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c67b8a-cbf2-45b5-aa1b-f718ea96b456 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:38.556879 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c67b8a-cbf2-45b5-aa1b-f718ea96b456 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:38.557155 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c67b8a-cbf2-45b5-aa1b-f718ea96b456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:38.586615 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-64c67b8a-cbf2-45b5-aa1b-f718ea96b456 None None] MFA Rules not processed for user `0ae8addce37444f3a629aa059a3efd9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:38.587874 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-64c67b8a-cbf2-45b5-aa1b-f718ea96b456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:38.655795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1039/2077] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:38.666901 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bad4b4-899d-477d-8ec5-51e521fbb222 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:38.668600 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-e9bad4b4-899d-477d-8ec5-51e521fbb222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:38.725731 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bad4b4-899d-477d-8ec5-51e521fbb222 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:38.729782 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45b3c10-cfa8-4973-b5d8-98b96a7e8722 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:38.730377 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b3c10-cfa8-4973-b5d8-98b96a7e8722 service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:38.730783 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b3c10-cfa8-4973-b5d8-98b96a7e8722 service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:38.731083 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b3c10-cfa8-4973-b5d8-98b96a7e8722 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:38.742597 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45b3c10-cfa8-4973-b5d8-98b96a7e8722 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:38.744069 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45b3c10-cfa8-4973-b5d8-98b96a7e8722 service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:38.754022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1039/2078] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:14:38 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:14:38.831778 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c67b8a-cbf2-45b5-aa1b-f718ea96b456 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:38.841155 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c67b8a-cbf2-45b5-aa1b-f718ea96b456 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:38.845301 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c32ece-7b46-4aa5-ae69-f90d2b7d2925 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:38.845885 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c32ece-7b46-4aa5-ae69-f90d2b7d2925 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:38.846157 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c32ece-7b46-4aa5-ae69-f90d2b7d2925 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:38.846421 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c32ece-7b46-4aa5-ae69-f90d2b7d2925 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:38.902564 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c32ece-7b46-4aa5-ae69-f90d2b7d2925 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:38.904011 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c32ece-7b46-4aa5-ae69-f90d2b7d2925 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:38.913295 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1040/2079] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:14:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:14:39.158792 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b2fad-2117-4e18-93d5-eebdac9afaaa None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:39.159317 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b2fad-2117-4e18-93d5-eebdac9afaaa None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:39.159431 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b2fad-2117-4e18-93d5-eebdac9afaaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:39.178764 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-4b5b2fad-2117-4e18-93d5-eebdac9afaaa None None] MFA Rules not processed for user `c7400d293884411b94618c5f3aac8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:39.180014 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-4b5b2fad-2117-4e18-93d5-eebdac9afaaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:39.218202 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:39.257265 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1040/2080] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:45.538974 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c32ece-7b46-4aa5-ae69-f90d2b7d2925 service cinder] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:45.540571 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-17c32ece-7b46-4aa5-ae69-f90d2b7d2925 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:45.634126 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c32ece-7b46-4aa5-ae69-f90d2b7d2925 service cinder] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:45.637195 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b960115c-3b79-4a0e-914a-d93505da841a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:45.637853 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b960115c-3b79-4a0e-914a-d93505da841a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:45.638131 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b960115c-3b79-4a0e-914a-d93505da841a admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:45.638399 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b960115c-3b79-4a0e-914a-d93505da841a admin admin] PATH_INFO: `/v3/users/27a50a253b3843d99747f432aa0e2cf5` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:45.639563 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b960115c-3b79-4a0e-914a-d93505da841a admin admin] RBAC: Authorizing `identity:delete_user(user_id=27a50a253b3843d99747f432aa0e2cf5)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:45.641453 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b960115c-3b79-4a0e-914a-d93505da841a admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:45.646485 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:45.695647 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b960115c-3b79-4a0e-914a-d93505da841a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27a50a253b3843d99747f432aa0e2cf5'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:45.709514 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b960115c-3b79-4a0e-914a-d93505da841a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27a50a253b3843d99747f432aa0e2cf5'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:45.710172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b960115c-3b79-4a0e-914a-d93505da841a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27a50a253b3843d99747f432aa0e2cf5'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:45.710776 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b960115c-3b79-4a0e-914a-d93505da841a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27a50a253b3843d99747f432aa0e2cf5'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:45.719702 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b960115c-3b79-4a0e-914a-d93505da841a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27a50a253b3843d99747f432aa0e2cf5'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:45.740014 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1041/2081] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:14:45 2022] DELETE /identity/v3/users/27a50a253b3843d99747f432aa0e2cf5 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:45.745817 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ad613-0468-4288-956a-f2b615c74346 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:45.746294 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ad613-0468-4288-956a-f2b615c74346 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:45.746713 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ad613-0468-4288-956a-f2b615c74346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:45.772720 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-ff8ad613-0468-4288-956a-f2b615c74346 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:45.773924 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-ff8ad613-0468-4288-956a-f2b615c74346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:45.854068 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1041/2082] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:45.866593 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b960115c-3b79-4a0e-914a-d93505da841a admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:45.868058 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b960115c-3b79-4a0e-914a-d93505da841a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:45.939603 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b960115c-3b79-4a0e-914a-d93505da841a admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:45.943378 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b3e2b-5d69-46ed-a0ae-8a1635112ede service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:45.944149 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b3e2b-5d69-46ed-a0ae-8a1635112ede service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:45.944376 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b3e2b-5d69-46ed-a0ae-8a1635112ede service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:45.945208 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b3e2b-5d69-46ed-a0ae-8a1635112ede service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:45.955832 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8b3e2b-5d69-46ed-a0ae-8a1635112ede service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:45.957255 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8b3e2b-5d69-46ed-a0ae-8a1635112ede service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:45.974202 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1042/2083] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:14:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:14:46.262795 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ad613-0468-4288-956a-f2b615c74346 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:46.264273 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-ff8ad613-0468-4288-956a-f2b615c74346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:46.276693 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ad613-0468-4288-956a-f2b615c74346 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:46.297586 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2192cbc1-81b5-448d-b5e7-70057f07d419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:46.298304 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192cbc1-81b5-448d-b5e7-70057f07d419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:46.298591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192cbc1-81b5-448d-b5e7-70057f07d419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:46.300146 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192cbc1-81b5-448d-b5e7-70057f07d419 admin admin] PATH_INFO: `/v3/projects/bbcad1815c724322942cbc57420cc919` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:46.302023 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2192cbc1-81b5-448d-b5e7-70057f07d419 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbcad1815c724322942cbc57420cc919)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:46.303985 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2192cbc1-81b5-448d-b5e7-70057f07d419 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:46.372322 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-2192cbc1-81b5-448d-b5e7-70057f07d419 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbcad1815c724322942cbc57420cc919', 'request_id': 'req-2192cbc1-81b5-448d-b5e7-70057f07d419'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:46.378882 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-2192cbc1-81b5-448d-b5e7-70057f07d419 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbcad1815c724322942cbc57420cc919', 'request_id': 'req-2192cbc1-81b5-448d-b5e7-70057f07d419'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:46.388062 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-2192cbc1-81b5-448d-b5e7-70057f07d419 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbcad1815c724322942cbc57420cc919', 'request_id': 'req-2192cbc1-81b5-448d-b5e7-70057f07d419'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:46.411595 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-2192cbc1-81b5-448d-b5e7-70057f07d419 admin admin] The token cache is being invalidate because project bbcad1815c724322942cbc57420cc919 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:14:46.411841 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-2192cbc1-81b5-448d-b5e7-70057f07d419 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:46.412605 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-2192cbc1-81b5-448d-b5e7-70057f07d419 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:46.414611 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1042/2084] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:14:46 2022] DELETE /identity/v3/projects/bbcad1815c724322942cbc57420cc919 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:46.419197 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e6df0-e4ec-40d5-bd20-4be25264153a None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:46.419489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e6df0-e4ec-40d5-bd20-4be25264153a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:46.419832 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e6df0-e4ec-40d5-bd20-4be25264153a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:46.449756 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-a03e6df0-e4ec-40d5-bd20-4be25264153a None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:46.449756 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-a03e6df0-e4ec-40d5-bd20-4be25264153a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:46.521518 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1043/2085] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:46.530545 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a9d28-5b25-4b28-97cd-4b6c3ae9bf26 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:46.530831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a9d28-5b25-4b28-97cd-4b6c3ae9bf26 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:46.531133 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a9d28-5b25-4b28-97cd-4b6c3ae9bf26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:46.554048 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-6c8a9d28-5b25-4b28-97cd-4b6c3ae9bf26 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:46.555286 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-6c8a9d28-5b25-4b28-97cd-4b6c3ae9bf26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:46.564233 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1043/2086] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:46.569579 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03e6df0-e4ec-40d5-bd20-4be25264153a None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:46.579912 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03e6df0-e4ec-40d5-bd20-4be25264153a None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:46.584526 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0566d-4bc8-4db0-b2c0-6d4c539ca36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:46.585121 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0566d-4bc8-4db0-b2c0-6d4c539ca36b admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:46.585400 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0566d-4bc8-4db0-b2c0-6d4c539ca36b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:46.585685 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0566d-4bc8-4db0-b2c0-6d4c539ca36b admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:46.586318 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e0566d-4bc8-4db0-b2c0-6d4c539ca36b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:14:46.586616 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e0566d-4bc8-4db0-b2c0-6d4c539ca36b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:46.588247 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e0566d-4bc8-4db0-b2c0-6d4c539ca36b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:46.601315 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1044/2087] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:14:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:14:46.605619 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a9d28-5b25-4b28-97cd-4b6c3ae9bf26 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:46.616015 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a9d28-5b25-4b28-97cd-4b6c3ae9bf26 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:46.620962 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0368bf-438e-4fd6-aa13-02878041569d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:46.621567 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0368bf-438e-4fd6-aa13-02878041569d admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:46.621847 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0368bf-438e-4fd6-aa13-02878041569d admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:46.622109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0368bf-438e-4fd6-aa13-02878041569d admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:46.622753 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0368bf-438e-4fd6-aa13-02878041569d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:46.624724 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0368bf-438e-4fd6-aa13-02878041569d admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:46.660768 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1044/2088] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:14:46 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:14:46.664716 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0566d-4bc8-4db0-b2c0-6d4c539ca36b admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:46.676430 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0566d-4bc8-4db0-b2c0-6d4c539ca36b admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:46.686310 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:46.735469 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c917d9e-481a-4b8d-8b02-44a7edd7b182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:46.736073 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c917d9e-481a-4b8d-8b02-44a7edd7b182 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:46.736351 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c917d9e-481a-4b8d-8b02-44a7edd7b182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:46.736622 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c917d9e-481a-4b8d-8b02-44a7edd7b182 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:46.737310 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c917d9e-481a-4b8d-8b02-44a7edd7b182 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:46.739351 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c917d9e-481a-4b8d-8b02-44a7edd7b182 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:46.767020 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1045/2089] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:14:46 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:14:46.768131 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0368bf-438e-4fd6-aa13-02878041569d admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:46.778698 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0368bf-438e-4fd6-aa13-02878041569d admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:46.783576 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280a8ff2-f20b-4730-8306-b1adc0dbcadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:46.784199 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a8ff2-f20b-4730-8306-b1adc0dbcadd admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:46.784508 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a8ff2-f20b-4730-8306-b1adc0dbcadd admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:46.784786 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a8ff2-f20b-4730-8306-b1adc0dbcadd admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:46.785438 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280a8ff2-f20b-4730-8306-b1adc0dbcadd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:46.787064 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280a8ff2-f20b-4730-8306-b1adc0dbcadd admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:46.796943 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1045/2090] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:14:46.800889 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c917d9e-481a-4b8d-8b02-44a7edd7b182 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:46.810705 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c917d9e-481a-4b8d-8b02-44a7edd7b182 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:46.816244 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd451f-6119-4072-9c12-1add55a9e621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:46.816862 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd451f-6119-4072-9c12-1add55a9e621 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:46.817144 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd451f-6119-4072-9c12-1add55a9e621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:46.817409 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd451f-6119-4072-9c12-1add55a9e621 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:46.818055 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acd451f-6119-4072-9c12-1add55a9e621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:46.819836 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acd451f-6119-4072-9c12-1add55a9e621 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:46.831814 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1046/2091] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:14:46.836089 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280a8ff2-f20b-4730-8306-b1adc0dbcadd admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:46.847076 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280a8ff2-f20b-4730-8306-b1adc0dbcadd admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:46.851986 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed982a8-4c6c-441f-93d3-9f216bdef4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:46.852700 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed982a8-4c6c-441f-93d3-9f216bdef4be admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:46.853036 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed982a8-4c6c-441f-93d3-9f216bdef4be admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:46.853332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed982a8-4c6c-441f-93d3-9f216bdef4be admin admin] PATH_INFO: `/v3/projects/f0220e0759444b29be6e9c2575914cc9/users/8342d909b3d24ec688c24e7df23a5c5d/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:46.870378 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed982a8-4c6c-441f-93d3-9f216bdef4be admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0220e0759444b29be6e9c2575914cc9, user_id=8342d909b3d24ec688c24e7df23a5c5d, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:46.873088 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed982a8-4c6c-441f-93d3-9f216bdef4be admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:46.887570 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1046/2092] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:14:46 2022] PUT /identity/v3/projects/f0220e0759444b29be6e9c2575914cc9/users/8342d909b3d24ec688c24e7df23a5c5d/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:14:46.898601 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c5b0f-7a06-46a9-9af0-8ccbd45261e1 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:46.898922 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c5b0f-7a06-46a9-9af0-8ccbd45261e1 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:46.899198 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c5b0f-7a06-46a9-9af0-8ccbd45261e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:46.926545 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-c18c5b0f-7a06-46a9-9af0-8ccbd45261e1 None None] MFA Rules not processed for user `8342d909b3d24ec688c24e7df23a5c5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:46.927792 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c18c5b0f-7a06-46a9-9af0-8ccbd45261e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:46.998742 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1047/2093] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:47.003634 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed982a8-4c6c-441f-93d3-9f216bdef4be admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:47.010012 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:47.073013 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed982a8-4c6c-441f-93d3-9f216bdef4be admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:47.075812 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62d7d7-4639-4be8-9c8a-384585992d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:47.076420 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62d7d7-4639-4be8-9c8a-384585992d07 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:47.076700 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62d7d7-4639-4be8-9c8a-384585992d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:47.076966 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62d7d7-4639-4be8-9c8a-384585992d07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:47.077614 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a62d7d7-4639-4be8-9c8a-384585992d07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:47.079534 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a62d7d7-4639-4be8-9c8a-384585992d07 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:47.105489 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1047/2094] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:14:47 2022] POST /identity/v3/projects => generated 381 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:14:47.110087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c5b0f-7a06-46a9-9af0-8ccbd45261e1 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:47.120769 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c5b0f-7a06-46a9-9af0-8ccbd45261e1 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:47.189127 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c5f05-9fdf-40b8-9708-385b12fb0358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:47.190015 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c5f05-9fdf-40b8-9708-385b12fb0358 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:47.190294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c5f05-9fdf-40b8-9708-385b12fb0358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:47.190561 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c5f05-9fdf-40b8-9708-385b12fb0358 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:47.191240 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776c5f05-9fdf-40b8-9708-385b12fb0358 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:47.193198 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776c5f05-9fdf-40b8-9708-385b12fb0358 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:47.215233 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1048/2095] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:14:47 2022] POST /identity/v3/users => generated 291 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:14:47.219229 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62d7d7-4639-4be8-9c8a-384585992d07 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:47.229488 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62d7d7-4639-4be8-9c8a-384585992d07 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:47.235344 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4940c8f4-35dd-4364-b6fd-865cfaf13be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:47.236069 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940c8f4-35dd-4364-b6fd-865cfaf13be4 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:47.236355 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940c8f4-35dd-4364-b6fd-865cfaf13be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:47.237028 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940c8f4-35dd-4364-b6fd-865cfaf13be4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:47.237742 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4940c8f4-35dd-4364-b6fd-865cfaf13be4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:47.239699 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4940c8f4-35dd-4364-b6fd-865cfaf13be4 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:47.251538 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1048/2096] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:14:47.255661 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c5f05-9fdf-40b8-9708-385b12fb0358 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:47.265878 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c5f05-9fdf-40b8-9708-385b12fb0358 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:47.270530 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7593e-0399-4a8d-bf04-640e043ba1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:47.271168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7593e-0399-4a8d-bf04-640e043ba1e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:47.271496 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7593e-0399-4a8d-bf04-640e043ba1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:47.271769 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7593e-0399-4a8d-bf04-640e043ba1e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:47.272408 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe7593e-0399-4a8d-bf04-640e043ba1e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:47.273984 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe7593e-0399-4a8d-bf04-640e043ba1e9 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:47.279303 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:47.284918 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1049/2097] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:14:47.289473 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4940c8f4-35dd-4364-b6fd-865cfaf13be4 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:47.304108 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4940c8f4-35dd-4364-b6fd-865cfaf13be4 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:47.308759 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4315cb-0a2c-4414-af3e-bc1843a67883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:47.309406 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4315cb-0a2c-4414-af3e-bc1843a67883 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:47.309679 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4315cb-0a2c-4414-af3e-bc1843a67883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:47.309952 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4315cb-0a2c-4414-af3e-bc1843a67883 admin admin] PATH_INFO: `/v3/projects/7f6bae56e2a54bfdab1b4c1f5fa30861/users/6e0e74b869254cc7835c2f66c1af8e9d/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:47.325690 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4315cb-0a2c-4414-af3e-bc1843a67883 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f6bae56e2a54bfdab1b4c1f5fa30861, user_id=6e0e74b869254cc7835c2f66c1af8e9d, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:47.328395 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4315cb-0a2c-4414-af3e-bc1843a67883 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:47.341455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1049/2098] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:14:47 2022] PUT /identity/v3/projects/7f6bae56e2a54bfdab1b4c1f5fa30861/users/6e0e74b869254cc7835c2f66c1af8e9d/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:14:47.355546 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8a4e9-dbd4-42db-93ba-26bf1565cbb6 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:47.355546 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8a4e9-dbd4-42db-93ba-26bf1565cbb6 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:47.355546 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8a4e9-dbd4-42db-93ba-26bf1565cbb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:47.382697 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-74e8a4e9-dbd4-42db-93ba-26bf1565cbb6 None None] MFA Rules not processed for user `6e0e74b869254cc7835c2f66c1af8e9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:47.383941 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-74e8a4e9-dbd4-42db-93ba-26bf1565cbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:47.451606 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1050/2099] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:47.456515 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4315cb-0a2c-4414-af3e-bc1843a67883 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:47.524906 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4315cb-0a2c-4414-af3e-bc1843a67883 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:47.533632 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec56588-3f37-4329-9b17-554d5fb864d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:47.534888 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec56588-3f37-4329-9b17-554d5fb864d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:47.535423 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec56588-3f37-4329-9b17-554d5fb864d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:47.536241 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec56588-3f37-4329-9b17-554d5fb864d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:47.536931 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec56588-3f37-4329-9b17-554d5fb864d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:47.540952 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec56588-3f37-4329-9b17-554d5fb864d3 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:47.575782 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1050/2100] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:14:47 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:14:47.580150 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8a4e9-dbd4-42db-93ba-26bf1565cbb6 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:47.597585 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8a4e9-dbd4-42db-93ba-26bf1565cbb6 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:47.667968 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66ea73-b76e-47b1-ac69-ee8870031942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:47.668585 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66ea73-b76e-47b1-ac69-ee8870031942 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:47.668872 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66ea73-b76e-47b1-ac69-ee8870031942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:47.669272 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66ea73-b76e-47b1-ac69-ee8870031942 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:47.669929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e66ea73-b76e-47b1-ac69-ee8870031942 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:47.672634 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e66ea73-b76e-47b1-ac69-ee8870031942 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:47.707824 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1051/2101] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:14:47 2022] POST /identity/v3/users => generated 298 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:14:47.715863 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec56588-3f37-4329-9b17-554d5fb864d3 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:47.726548 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec56588-3f37-4329-9b17-554d5fb864d3 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:47.731232 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe9645-51b9-4cd0-83d3-97d6bf41f87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:47.731830 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe9645-51b9-4cd0-83d3-97d6bf41f87b admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:47.732109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe9645-51b9-4cd0-83d3-97d6bf41f87b admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:47.732376 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe9645-51b9-4cd0-83d3-97d6bf41f87b admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:47.733653 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fe9645-51b9-4cd0-83d3-97d6bf41f87b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:47.735259 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fe9645-51b9-4cd0-83d3-97d6bf41f87b admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:47.747036 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1051/2102] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:14:47.751516 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66ea73-b76e-47b1-ac69-ee8870031942 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:47.761647 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66ea73-b76e-47b1-ac69-ee8870031942 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:47.767914 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876ab39-739d-4a08-9ca5-83445186a230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:47.768660 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876ab39-739d-4a08-9ca5-83445186a230 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:47.769016 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876ab39-739d-4a08-9ca5-83445186a230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:47.769358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876ab39-739d-4a08-9ca5-83445186a230 admin admin] PATH_INFO: `/v3/projects/f92ad073da6641f4add6bfd711eb6a2c/users/dcac7720c9ed4cbba87b51a183efa514/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:47.787625 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e876ab39-739d-4a08-9ca5-83445186a230 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f92ad073da6641f4add6bfd711eb6a2c, user_id=dcac7720c9ed4cbba87b51a183efa514, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:47.790257 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e876ab39-739d-4a08-9ca5-83445186a230 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:47.808941 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1052/2103] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:14:47 2022] PUT /identity/v3/projects/f92ad073da6641f4add6bfd711eb6a2c/users/dcac7720c9ed4cbba87b51a183efa514/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:47.813063 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe9645-51b9-4cd0-83d3-97d6bf41f87b admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:47.824329 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe9645-51b9-4cd0-83d3-97d6bf41f87b admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:47.854514 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:47.885387 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2df203-53c4-4b06-b6df-be477f6d9bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:47.885992 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2df203-53c4-4b06-b6df-be477f6d9bed admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:47.886269 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2df203-53c4-4b06-b6df-be477f6d9bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:47.886540 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2df203-53c4-4b06-b6df-be477f6d9bed admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:47.887210 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2df203-53c4-4b06-b6df-be477f6d9bed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:47.888793 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2df203-53c4-4b06-b6df-be477f6d9bed admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:47.901189 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1052/2104] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:14:47.904005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876ab39-739d-4a08-9ca5-83445186a230 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:47.915160 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876ab39-739d-4a08-9ca5-83445186a230 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:47.920279 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39907a53-6c1d-4834-98ff-e8de268decc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:47.921028 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39907a53-6c1d-4834-98ff-e8de268decc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:47.921313 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39907a53-6c1d-4834-98ff-e8de268decc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:47.921584 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39907a53-6c1d-4834-98ff-e8de268decc1 admin admin] PATH_INFO: `/v3/projects/f92ad073da6641f4add6bfd711eb6a2c/users/dcac7720c9ed4cbba87b51a183efa514/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:47.923616 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39907a53-6c1d-4834-98ff-e8de268decc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f92ad073da6641f4add6bfd711eb6a2c, user_id=dcac7720c9ed4cbba87b51a183efa514, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:47.926436 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39907a53-6c1d-4834-98ff-e8de268decc1 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:47.939073 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1053/2105] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:14:47 2022] PUT /identity/v3/projects/f92ad073da6641f4add6bfd711eb6a2c/users/dcac7720c9ed4cbba87b51a183efa514/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:47.950106 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f70ff8-fbab-486a-8c9b-f1e53e497424 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:47.950394 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f70ff8-fbab-486a-8c9b-f1e53e497424 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:47.950671 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f70ff8-fbab-486a-8c9b-f1e53e497424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:47.981538 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-22f70ff8-fbab-486a-8c9b-f1e53e497424 None None] MFA Rules not processed for user `dcac7720c9ed4cbba87b51a183efa514`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:47.982965 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-22f70ff8-fbab-486a-8c9b-f1e53e497424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:48.060974 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1053/2106] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:47 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:48.073151 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39907a53-6c1d-4834-98ff-e8de268decc1 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:48.075028 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-39907a53-6c1d-4834-98ff-e8de268decc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:48.130408 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:48.142908 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39907a53-6c1d-4834-98ff-e8de268decc1 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:48.148006 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253598a0-eb05-4e4e-96ad-848008945ec2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:48.148934 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253598a0-eb05-4e4e-96ad-848008945ec2 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:48.149362 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253598a0-eb05-4e4e-96ad-848008945ec2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:48.149910 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253598a0-eb05-4e4e-96ad-848008945ec2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:48.210717 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253598a0-eb05-4e4e-96ad-848008945ec2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:48.212179 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253598a0-eb05-4e4e-96ad-848008945ec2 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:48.221835 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1054/2107] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:14:48 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:14:49.737806 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f70ff8-fbab-486a-8c9b-f1e53e497424 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:49.747613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f70ff8-fbab-486a-8c9b-f1e53e497424 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:49.751212 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364b85bc-ed07-4e0e-b872-2dde1e591a68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:49.751806 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b85bc-ed07-4e0e-b872-2dde1e591a68 service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:49.752083 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b85bc-ed07-4e0e-b872-2dde1e591a68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:49.752356 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b85bc-ed07-4e0e-b872-2dde1e591a68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:49.814130 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364b85bc-ed07-4e0e-b872-2dde1e591a68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:49.815597 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364b85bc-ed07-4e0e-b872-2dde1e591a68 service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:49.825041 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1054/2108] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:14:49 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:14:50.404294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253598a0-eb05-4e4e-96ad-848008945ec2 service cinder] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:50.418047 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253598a0-eb05-4e4e-96ad-848008945ec2 service cinder] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:50.422596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9caf2b-1c9b-44a4-800a-48879b0ae864 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:50.423288 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9caf2b-1c9b-44a4-800a-48879b0ae864 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:50.423705 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9caf2b-1c9b-44a4-800a-48879b0ae864 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:50.423888 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9caf2b-1c9b-44a4-800a-48879b0ae864 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:50.440917 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9caf2b-1c9b-44a4-800a-48879b0ae864 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:50.442344 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9caf2b-1c9b-44a4-800a-48879b0ae864 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:50.462852 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1055/2109] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:14:50 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:14:54.440418 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364b85bc-ed07-4e0e-b872-2dde1e591a68 service cinder] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:54.495248 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:54.512672 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364b85bc-ed07-4e0e-b872-2dde1e591a68 service cinder] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:54.517862 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6977dd0-2944-4dbc-a898-41c114383527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:54.518576 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6977dd0-2944-4dbc-a898-41c114383527 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:54.518877 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6977dd0-2944-4dbc-a898-41c114383527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:54.519150 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6977dd0-2944-4dbc-a898-41c114383527 admin admin] PATH_INFO: `/v3/users/8342d909b3d24ec688c24e7df23a5c5d` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:54.520325 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6977dd0-2944-4dbc-a898-41c114383527 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8342d909b3d24ec688c24e7df23a5c5d)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:54.522251 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6977dd0-2944-4dbc-a898-41c114383527 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:54.576997 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-e6977dd0-2944-4dbc-a898-41c114383527 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8342d909b3d24ec688c24e7df23a5c5d'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:54.585083 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-e6977dd0-2944-4dbc-a898-41c114383527 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8342d909b3d24ec688c24e7df23a5c5d'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:54.586000 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-e6977dd0-2944-4dbc-a898-41c114383527 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8342d909b3d24ec688c24e7df23a5c5d'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:54.586908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-e6977dd0-2944-4dbc-a898-41c114383527 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8342d909b3d24ec688c24e7df23a5c5d'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:54.595084 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-e6977dd0-2944-4dbc-a898-41c114383527 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8342d909b3d24ec688c24e7df23a5c5d'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:54.613558 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1055/2110] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:14:54 2022] DELETE /identity/v3/users/8342d909b3d24ec688c24e7df23a5c5d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:54.618442 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f78162-3564-49ef-bef7-64d21ff52cea None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:54.618953 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f78162-3564-49ef-bef7-64d21ff52cea None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:54.619233 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f78162-3564-49ef-bef7-64d21ff52cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:54.638734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-80f78162-3564-49ef-bef7-64d21ff52cea None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:14:54.640541 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-80f78162-3564-49ef-bef7-64d21ff52cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:54.702414 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1056/2111] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:14:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:14:54.712255 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6977dd0-2944-4dbc-a898-41c114383527 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:54.713709 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-e6977dd0-2944-4dbc-a898-41c114383527 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:54.779855 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6977dd0-2944-4dbc-a898-41c114383527 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:54.784176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb77a31-8312-424f-9ff2-a5e1f62f23aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:54.784869 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb77a31-8312-424f-9ff2-a5e1f62f23aa service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:54.785151 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb77a31-8312-424f-9ff2-a5e1f62f23aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:54.785545 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb77a31-8312-424f-9ff2-a5e1f62f23aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:54.798522 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb77a31-8312-424f-9ff2-a5e1f62f23aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:54.800159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb77a31-8312-424f-9ff2-a5e1f62f23aa service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:54.813305 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1056/2112] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:14:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:14:55.092586 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f78162-3564-49ef-bef7-64d21ff52cea None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:55.094043 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-80f78162-3564-49ef-bef7-64d21ff52cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:55.108221 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f78162-3564-49ef-bef7-64d21ff52cea None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:55.113161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868cd39-0cae-4c87-8e87-5193a169eaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:55.113801 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868cd39-0cae-4c87-8e87-5193a169eaa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:55.114085 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868cd39-0cae-4c87-8e87-5193a169eaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:55.114346 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868cd39-0cae-4c87-8e87-5193a169eaa1 admin admin] PATH_INFO: `/v3/projects/f0220e0759444b29be6e9c2575914cc9` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:55.124512 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b868cd39-0cae-4c87-8e87-5193a169eaa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0220e0759444b29be6e9c2575914cc9)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:55.126422 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b868cd39-0cae-4c87-8e87-5193a169eaa1 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:55.173708 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b868cd39-0cae-4c87-8e87-5193a169eaa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0220e0759444b29be6e9c2575914cc9', 'request_id': 'req-b868cd39-0cae-4c87-8e87-5193a169eaa1'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:55.182137 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b868cd39-0cae-4c87-8e87-5193a169eaa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0220e0759444b29be6e9c2575914cc9', 'request_id': 'req-b868cd39-0cae-4c87-8e87-5193a169eaa1'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:55.188382 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b868cd39-0cae-4c87-8e87-5193a169eaa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0220e0759444b29be6e9c2575914cc9', 'request_id': 'req-b868cd39-0cae-4c87-8e87-5193a169eaa1'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:55.199847 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b868cd39-0cae-4c87-8e87-5193a169eaa1 admin admin] The token cache is being invalidate because project f0220e0759444b29be6e9c2575914cc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:14:55.200164 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b868cd39-0cae-4c87-8e87-5193a169eaa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:55.200874 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b868cd39-0cae-4c87-8e87-5193a169eaa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:55.203230 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1057/2113] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:14:55 2022] DELETE /identity/v3/projects/f0220e0759444b29be6e9c2575914cc9 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:55.206992 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb77a31-8312-424f-9ff2-a5e1f62f23aa service neutron] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:55.216592 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb77a31-8312-424f-9ff2-a5e1f62f23aa service neutron] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:55.278924 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec246a-60cb-45fe-ae34-8437402b0b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:55.279522 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec246a-60cb-45fe-ae34-8437402b0b43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:55.279826 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec246a-60cb-45fe-ae34-8437402b0b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:55.280097 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec246a-60cb-45fe-ae34-8437402b0b43 admin admin] PATH_INFO: `/v3/users/6e0e74b869254cc7835c2f66c1af8e9d` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:55.281246 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ec246a-60cb-45fe-ae34-8437402b0b43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e0e74b869254cc7835c2f66c1af8e9d)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:55.283594 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ec246a-60cb-45fe-ae34-8437402b0b43 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:55.297653 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:55.343905 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-e9ec246a-60cb-45fe-ae34-8437402b0b43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e0e74b869254cc7835c2f66c1af8e9d'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:55.352951 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-e9ec246a-60cb-45fe-ae34-8437402b0b43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e0e74b869254cc7835c2f66c1af8e9d'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:55.353619 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-e9ec246a-60cb-45fe-ae34-8437402b0b43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e0e74b869254cc7835c2f66c1af8e9d'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:55.354240 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-e9ec246a-60cb-45fe-ae34-8437402b0b43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e0e74b869254cc7835c2f66c1af8e9d'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:55.363030 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-e9ec246a-60cb-45fe-ae34-8437402b0b43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e0e74b869254cc7835c2f66c1af8e9d'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:55.384632 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1057/2114] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:14:55 2022] DELETE /identity/v3/users/6e0e74b869254cc7835c2f66c1af8e9d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:55.672949 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868cd39-0cae-4c87-8e87-5193a169eaa1 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:55.674401 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b868cd39-0cae-4c87-8e87-5193a169eaa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:55.691092 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868cd39-0cae-4c87-8e87-5193a169eaa1 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:55.695531 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:55.751772 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38dd2be-05d3-46a3-968d-cbaa2d641eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:55.752401 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38dd2be-05d3-46a3-968d-cbaa2d641eb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:55.752684 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38dd2be-05d3-46a3-968d-cbaa2d641eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:55.752965 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38dd2be-05d3-46a3-968d-cbaa2d641eb8 admin admin] PATH_INFO: `/v3/projects/7f6bae56e2a54bfdab1b4c1f5fa30861` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:55.754069 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38dd2be-05d3-46a3-968d-cbaa2d641eb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f6bae56e2a54bfdab1b4c1f5fa30861)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:55.756029 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38dd2be-05d3-46a3-968d-cbaa2d641eb8 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:55.805300 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e38dd2be-05d3-46a3-968d-cbaa2d641eb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f6bae56e2a54bfdab1b4c1f5fa30861', 'request_id': 'req-e38dd2be-05d3-46a3-968d-cbaa2d641eb8'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:55.812221 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e38dd2be-05d3-46a3-968d-cbaa2d641eb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f6bae56e2a54bfdab1b4c1f5fa30861', 'request_id': 'req-e38dd2be-05d3-46a3-968d-cbaa2d641eb8'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:55.817927 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e38dd2be-05d3-46a3-968d-cbaa2d641eb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f6bae56e2a54bfdab1b4c1f5fa30861', 'request_id': 'req-e38dd2be-05d3-46a3-968d-cbaa2d641eb8'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:55.829922 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e38dd2be-05d3-46a3-968d-cbaa2d641eb8 admin admin] The token cache is being invalidate because project 7f6bae56e2a54bfdab1b4c1f5fa30861 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:14:55.830237 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e38dd2be-05d3-46a3-968d-cbaa2d641eb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:55.830908 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e38dd2be-05d3-46a3-968d-cbaa2d641eb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:55.832395 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1058/2115] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:14:55 2022] DELETE /identity/v3/projects/7f6bae56e2a54bfdab1b4c1f5fa30861 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:55.836532 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec246a-60cb-45fe-ae34-8437402b0b43 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:55.838047 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-e9ec246a-60cb-45fe-ae34-8437402b0b43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:55.851324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec246a-60cb-45fe-ae34-8437402b0b43 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:55.906485 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a193e1bf-d56b-473b-b6e8-a08eacd6986c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:55.907152 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193e1bf-d56b-473b-b6e8-a08eacd6986c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:55.907427 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193e1bf-d56b-473b-b6e8-a08eacd6986c admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:55.907696 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193e1bf-d56b-473b-b6e8-a08eacd6986c admin admin] PATH_INFO: `/v3/users/dcac7720c9ed4cbba87b51a183efa514` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:55.908826 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a193e1bf-d56b-473b-b6e8-a08eacd6986c admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcac7720c9ed4cbba87b51a183efa514)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:55.910714 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a193e1bf-d56b-473b-b6e8-a08eacd6986c admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:55.962027 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-a193e1bf-d56b-473b-b6e8-a08eacd6986c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcac7720c9ed4cbba87b51a183efa514'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:55.969568 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-a193e1bf-d56b-473b-b6e8-a08eacd6986c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcac7720c9ed4cbba87b51a183efa514'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:55.970190 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-a193e1bf-d56b-473b-b6e8-a08eacd6986c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcac7720c9ed4cbba87b51a183efa514'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:55.970879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-a193e1bf-d56b-473b-b6e8-a08eacd6986c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcac7720c9ed4cbba87b51a183efa514'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:55.977951 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-a193e1bf-d56b-473b-b6e8-a08eacd6986c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcac7720c9ed4cbba87b51a183efa514'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:55.995244 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1058/2116] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:14:55 2022] DELETE /identity/v3/users/dcac7720c9ed4cbba87b51a183efa514 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:14:56.257721 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38dd2be-05d3-46a3-968d-cbaa2d641eb8 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:14:56.259304 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e38dd2be-05d3-46a3-968d-cbaa2d641eb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:14:56.273750 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38dd2be-05d3-46a3-968d-cbaa2d641eb8 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:14:56.330720 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ae5c5-9afe-4364-8a2e-cb3dd9700bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:14:56.331382 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ae5c5-9afe-4364-8a2e-cb3dd9700bf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:14:56.331663 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ae5c5-9afe-4364-8a2e-cb3dd9700bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:14:56.331935 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ae5c5-9afe-4364-8a2e-cb3dd9700bf8 admin admin] PATH_INFO: `/v3/projects/f92ad073da6641f4add6bfd711eb6a2c` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:14:56.333039 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75ae5c5-9afe-4364-8a2e-cb3dd9700bf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f92ad073da6641f4add6bfd711eb6a2c)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:14:56.334952 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75ae5c5-9afe-4364-8a2e-cb3dd9700bf8 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:14:56.373818 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:14:56.382507 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f75ae5c5-9afe-4364-8a2e-cb3dd9700bf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f92ad073da6641f4add6bfd711eb6a2c', 'request_id': 'req-f75ae5c5-9afe-4364-8a2e-cb3dd9700bf8'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:56.390611 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f75ae5c5-9afe-4364-8a2e-cb3dd9700bf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f92ad073da6641f4add6bfd711eb6a2c', 'request_id': 'req-f75ae5c5-9afe-4364-8a2e-cb3dd9700bf8'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:56.396855 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f75ae5c5-9afe-4364-8a2e-cb3dd9700bf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f92ad073da6641f4add6bfd711eb6a2c', 'request_id': 'req-f75ae5c5-9afe-4364-8a2e-cb3dd9700bf8'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:56.408462 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f75ae5c5-9afe-4364-8a2e-cb3dd9700bf8 admin admin] The token cache is being invalidate because project f92ad073da6641f4add6bfd711eb6a2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:14:56.408755 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f75ae5c5-9afe-4364-8a2e-cb3dd9700bf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:56.409391 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f75ae5c5-9afe-4364-8a2e-cb3dd9700bf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:14:56.410889 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1059/2117] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:14:56 2022] DELETE /identity/v3/projects/f92ad073da6641f4add6bfd711eb6a2c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:15:03.991061 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a4e85-e05f-4db0-8f8d-ca28cf0b5379 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:03.991540 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a4e85-e05f-4db0-8f8d-ca28cf0b5379 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:03.991654 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a4e85-e05f-4db0-8f8d-ca28cf0b5379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:04.011755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-7b3a4e85-e05f-4db0-8f8d-ca28cf0b5379 None None] MFA Rules not processed for user `1405e758a3af4e8ba659f498cd28ace1`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:15:04.012945 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-7b3a4e85-e05f-4db0-8f8d-ca28cf0b5379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:04.017828 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:15:04.079634 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1059/2118] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Sep 12 14:15:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:15:04.125706 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ae5c5-9afe-4364-8a2e-cb3dd9700bf8 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:04.127416 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f75ae5c5-9afe-4364-8a2e-cb3dd9700bf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:04.199302 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ae5c5-9afe-4364-8a2e-cb3dd9700bf8 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:04.204270 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b217bae-7573-443d-b78d-597e163462b8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a87ea9c3c5409b97a35c4857ae3ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:04.204871 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b217bae-7573-443d-b78d-597e163462b8 service swift] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:04.205150 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b217bae-7573-443d-b78d-597e163462b8 service swift] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:04.205421 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b217bae-7573-443d-b78d-597e163462b8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:04.213225 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b217bae-7573-443d-b78d-597e163462b8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:04.214780 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b217bae-7573-443d-b78d-597e163462b8 service swift] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:04.223234 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1060/2119] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Sep 12 14:15:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 14:15:04.740122 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f4d962-41c3-405f-b171-b2b30e4429bb None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:04.740431 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f4d962-41c3-405f-b171-b2b30e4429bb None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:04.740708 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f4d962-41c3-405f-b171-b2b30e4429bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:04.760168 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-a1f4d962-41c3-405f-b171-b2b30e4429bb None None] MFA Rules not processed for user `1405e758a3af4e8ba659f498cd28ace1`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:15:04.761349 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-a1f4d962-41c3-405f-b171-b2b30e4429bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:04.769049 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1060/2120] 10.222.0.42 () {58 vars in 967 bytes} [Mon Sep 12 14:15:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:15:04.781440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b217bae-7573-443d-b78d-597e163462b8 service swift] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:04.791598 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b217bae-7573-443d-b78d-597e163462b8 service swift] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:04.796527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7f23b-8896-4e88-8892-9dad0f979f87 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a87ea9c3c5409b97a35c4857ae3ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:04.797103 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7f23b-8896-4e88-8892-9dad0f979f87 service swift] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:04.797379 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7f23b-8896-4e88-8892-9dad0f979f87 service swift] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:04.797650 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7f23b-8896-4e88-8892-9dad0f979f87 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:04.805430 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc7f23b-8896-4e88-8892-9dad0f979f87 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:04.807364 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc7f23b-8896-4e88-8892-9dad0f979f87 service swift] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:04.815847 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1061/2121] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Sep 12 14:15:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 12 14:15:05.060429 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c00c0c-5991-4461-b69b-81f262db1931 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:05.060910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c00c0c-5991-4461-b69b-81f262db1931 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:05.061014 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c00c0c-5991-4461-b69b-81f262db1931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:05.080436 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-84c00c0c-5991-4461-b69b-81f262db1931 None None] MFA Rules not processed for user `1405e758a3af4e8ba659f498cd28ace1`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:15:05.082370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-84c00c0c-5991-4461-b69b-81f262db1931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:05.092058 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1061/2122] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Sep 12 14:15:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:15:05.104696 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7f23b-8896-4e88-8892-9dad0f979f87 service swift] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:05.115144 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7f23b-8896-4e88-8892-9dad0f979f87 service swift] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:05.120296 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da791d3d-0c09-48e9-9bec-fbff9866e45d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a87ea9c3c5409b97a35c4857ae3ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:05.120887 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da791d3d-0c09-48e9-9bec-fbff9866e45d service swift] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:05.121167 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da791d3d-0c09-48e9-9bec-fbff9866e45d service swift] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:05.121467 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da791d3d-0c09-48e9-9bec-fbff9866e45d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:05.129466 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da791d3d-0c09-48e9-9bec-fbff9866e45d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:05.131051 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da791d3d-0c09-48e9-9bec-fbff9866e45d service swift] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:05.139300 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1062/2123] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Sep 12 14:15:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 14:15:05.473362 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3bfb60-891c-4e90-ad87-19a611ec8e39 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:05.473653 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3bfb60-891c-4e90-ad87-19a611ec8e39 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:05.473930 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3bfb60-891c-4e90-ad87-19a611ec8e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:05.493253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-8b3bfb60-891c-4e90-ad87-19a611ec8e39 None None] MFA Rules not processed for user `1405e758a3af4e8ba659f498cd28ace1`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:15:05.494441 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-8b3bfb60-891c-4e90-ad87-19a611ec8e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:05.501613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1062/2124] 10.222.0.42 () {58 vars in 967 bytes} [Mon Sep 12 14:15:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:15:05.513771 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da791d3d-0c09-48e9-9bec-fbff9866e45d service swift] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:05.524124 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da791d3d-0c09-48e9-9bec-fbff9866e45d service swift] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:05.529119 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84fda3-7f36-4895-9318-f021db3da4f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a87ea9c3c5409b97a35c4857ae3ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:05.529706 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84fda3-7f36-4895-9318-f021db3da4f5 service swift] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:05.530518 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84fda3-7f36-4895-9318-f021db3da4f5 service swift] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:05.530766 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84fda3-7f36-4895-9318-f021db3da4f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:05.538567 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca84fda3-7f36-4895-9318-f021db3da4f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:05.540147 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca84fda3-7f36-4895-9318-f021db3da4f5 service swift] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:05.548458 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1063/2125] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Sep 12 14:15:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 14:15:11.832779 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3bfb60-891c-4e90-ad87-19a611ec8e39 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:11.834284 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-8b3bfb60-891c-4e90-ad87-19a611ec8e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:11.901882 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3bfb60-891c-4e90-ad87-19a611ec8e39 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:11.906379 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d4eed-13d1-454b-9b5a-d9eea1d4e7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:11.907033 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d4eed-13d1-454b-9b5a-d9eea1d4e7fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:11.907307 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d4eed-13d1-454b-9b5a-d9eea1d4e7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:11.907574 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d4eed-13d1-454b-9b5a-d9eea1d4e7fe admin admin] PATH_INFO: `/v3/users/c7400d293884411b94618c5f3aac8c43` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:11.908684 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d4eed-13d1-454b-9b5a-d9eea1d4e7fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7400d293884411b94618c5f3aac8c43)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:11.910616 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d4eed-13d1-454b-9b5a-d9eea1d4e7fe admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:11.957920 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-495d4eed-13d1-454b-9b5a-d9eea1d4e7fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7400d293884411b94618c5f3aac8c43'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:11.966471 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-495d4eed-13d1-454b-9b5a-d9eea1d4e7fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7400d293884411b94618c5f3aac8c43'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:11.967336 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-495d4eed-13d1-454b-9b5a-d9eea1d4e7fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7400d293884411b94618c5f3aac8c43'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:11.967954 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-495d4eed-13d1-454b-9b5a-d9eea1d4e7fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7400d293884411b94618c5f3aac8c43'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:11.976062 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-495d4eed-13d1-454b-9b5a-d9eea1d4e7fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7400d293884411b94618c5f3aac8c43'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:12.000036 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1063/2126] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:15:11 2022] DELETE /identity/v3/users/c7400d293884411b94618c5f3aac8c43 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:15:12.005150 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3f1bf-a6de-4a6b-82f9-3d6017a7d5a1 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:12.005664 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3f1bf-a6de-4a6b-82f9-3d6017a7d5a1 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:12.005785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3f1bf-a6de-4a6b-82f9-3d6017a7d5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:12.027242 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-d8b3f1bf-a6de-4a6b-82f9-3d6017a7d5a1 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:15:12.028505 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d8b3f1bf-a6de-4a6b-82f9-3d6017a7d5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:12.100583 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1064/2127] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:15:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:15:12.111836 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d4eed-13d1-454b-9b5a-d9eea1d4e7fe admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:12.113429 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-495d4eed-13d1-454b-9b5a-d9eea1d4e7fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:12.136735 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:15:12.200028 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d4eed-13d1-454b-9b5a-d9eea1d4e7fe admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:12.207249 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e2d20-c8b2-4773-8493-98f7ba72a89c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:12.208303 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e2d20-c8b2-4773-8493-98f7ba72a89c service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:12.208786 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e2d20-c8b2-4773-8493-98f7ba72a89c service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:12.209059 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e2d20-c8b2-4773-8493-98f7ba72a89c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:12.225162 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01e2d20-c8b2-4773-8493-98f7ba72a89c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:12.226601 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01e2d20-c8b2-4773-8493-98f7ba72a89c service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:12.238269 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1064/2128] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:15:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:15:12.413222 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b3f1bf-a6de-4a6b-82f9-3d6017a7d5a1 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:12.415216 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d8b3f1bf-a6de-4a6b-82f9-3d6017a7d5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:12.429019 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b3f1bf-a6de-4a6b-82f9-3d6017a7d5a1 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:12.434409 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230dab5-ab2d-4c52-99f8-2a4f01abcd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:12.435085 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230dab5-ab2d-4c52-99f8-2a4f01abcd4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:12.435367 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230dab5-ab2d-4c52-99f8-2a4f01abcd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:12.435640 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230dab5-ab2d-4c52-99f8-2a4f01abcd4d admin admin] PATH_INFO: `/v3/projects/7f946cd56d394d75935e5277811c39c7` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:12.436757 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230dab5-ab2d-4c52-99f8-2a4f01abcd4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f946cd56d394d75935e5277811c39c7)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:12.438984 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230dab5-ab2d-4c52-99f8-2a4f01abcd4d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:12.450681 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:15:12.490393 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-5230dab5-ab2d-4c52-99f8-2a4f01abcd4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f946cd56d394d75935e5277811c39c7', 'request_id': 'req-5230dab5-ab2d-4c52-99f8-2a4f01abcd4d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:12.496855 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-5230dab5-ab2d-4c52-99f8-2a4f01abcd4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f946cd56d394d75935e5277811c39c7', 'request_id': 'req-5230dab5-ab2d-4c52-99f8-2a4f01abcd4d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:12.506240 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-5230dab5-ab2d-4c52-99f8-2a4f01abcd4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f946cd56d394d75935e5277811c39c7', 'request_id': 'req-5230dab5-ab2d-4c52-99f8-2a4f01abcd4d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:12.521897 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-5230dab5-ab2d-4c52-99f8-2a4f01abcd4d admin admin] The token cache is being invalidate because project 7f946cd56d394d75935e5277811c39c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:15:12.522197 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-5230dab5-ab2d-4c52-99f8-2a4f01abcd4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:12.522931 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-5230dab5-ab2d-4c52-99f8-2a4f01abcd4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:12.524356 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1065/2129] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:15:12 2022] DELETE /identity/v3/projects/7f946cd56d394d75935e5277811c39c7 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:15:12.529232 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a0431-ca0a-46b4-b944-db0e65c2725e None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:12.529564 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a0431-ca0a-46b4-b944-db0e65c2725e None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:12.530472 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a0431-ca0a-46b4-b944-db0e65c2725e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:12.549478 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-956a0431-ca0a-46b4-b944-db0e65c2725e None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:15:12.550648 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-956a0431-ca0a-46b4-b944-db0e65c2725e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:12.610001 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1065/2130] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:15:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:15:12.618192 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169176da-bf90-4090-883f-e2c24088ed7b None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:12.618519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169176da-bf90-4090-883f-e2c24088ed7b None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:12.618793 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169176da-bf90-4090-883f-e2c24088ed7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:12.638287 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-169176da-bf90-4090-883f-e2c24088ed7b None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:15:12.640024 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-169176da-bf90-4090-883f-e2c24088ed7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:12.647779 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1066/2131] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:15:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:15:12.651968 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a0431-ca0a-46b4-b944-db0e65c2725e None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:12.662098 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a0431-ca0a-46b4-b944-db0e65c2725e None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:12.666644 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813350c1-b589-4597-a8d9-c59a09245a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:12.667260 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813350c1-b589-4597-a8d9-c59a09245a3a admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:12.667539 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813350c1-b589-4597-a8d9-c59a09245a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:12.667808 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813350c1-b589-4597-a8d9-c59a09245a3a admin admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:12.668442 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813350c1-b589-4597-a8d9-c59a09245a3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:15:12.668741 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813350c1-b589-4597-a8d9-c59a09245a3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:12.670312 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813350c1-b589-4597-a8d9-c59a09245a3a admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:12.682953 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1066/2132] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:15:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:15:12.687193 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169176da-bf90-4090-883f-e2c24088ed7b None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:12.696469 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169176da-bf90-4090-883f-e2c24088ed7b None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:12.701651 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167735d8-0fb6-47db-9f31-221c5466864c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:12.702539 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167735d8-0fb6-47db-9f31-221c5466864c admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:12.702539 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167735d8-0fb6-47db-9f31-221c5466864c admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:12.702798 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167735d8-0fb6-47db-9f31-221c5466864c admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:12.703452 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167735d8-0fb6-47db-9f31-221c5466864c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:12.705327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167735d8-0fb6-47db-9f31-221c5466864c admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:12.736942 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1067/2133] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:15:12 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:15:12.738430 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813350c1-b589-4597-a8d9-c59a09245a3a admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:12.748429 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813350c1-b589-4597-a8d9-c59a09245a3a admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:12.805173 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48069c3a-c8bb-4ed9-8ecd-697ad828cfb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:12.805700 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48069c3a-c8bb-4ed9-8ecd-697ad828cfb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:12.805979 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48069c3a-c8bb-4ed9-8ecd-697ad828cfb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:12.806248 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48069c3a-c8bb-4ed9-8ecd-697ad828cfb0 admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:12.806898 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48069c3a-c8bb-4ed9-8ecd-697ad828cfb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:12.808792 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48069c3a-c8bb-4ed9-8ecd-697ad828cfb0 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:12.838417 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1067/2134] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:15:12 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:15:12.839686 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167735d8-0fb6-47db-9f31-221c5466864c admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:12.850410 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167735d8-0fb6-47db-9f31-221c5466864c admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:12.855212 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae39969-1c3d-4cf0-ad42-d56722488cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:12.855814 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae39969-1c3d-4cf0-ad42-d56722488cef admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:12.856093 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae39969-1c3d-4cf0-ad42-d56722488cef admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:12.856362 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae39969-1c3d-4cf0-ad42-d56722488cef admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:12.856996 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae39969-1c3d-4cf0-ad42-d56722488cef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:12.859137 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae39969-1c3d-4cf0-ad42-d56722488cef admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:12.873184 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1068/2135] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:15:12.875044 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48069c3a-c8bb-4ed9-8ecd-697ad828cfb0 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:12.885306 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48069c3a-c8bb-4ed9-8ecd-697ad828cfb0 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:12.890266 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc9a3e-5bd1-4098-9542-218875ef5068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:12.890881 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc9a3e-5bd1-4098-9542-218875ef5068 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:12.891161 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc9a3e-5bd1-4098-9542-218875ef5068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:12.891423 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc9a3e-5bd1-4098-9542-218875ef5068 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:12.892096 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc9a3e-5bd1-4098-9542-218875ef5068 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:12.893609 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc9a3e-5bd1-4098-9542-218875ef5068 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:12.904638 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1068/2136] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:15:12.909524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae39969-1c3d-4cf0-ad42-d56722488cef admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:12.919331 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae39969-1c3d-4cf0-ad42-d56722488cef admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:12.924757 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc448d7e-0577-4f62-a701-84c8a5a0873a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:12.925392 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc448d7e-0577-4f62-a701-84c8a5a0873a admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:12.925673 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc448d7e-0577-4f62-a701-84c8a5a0873a admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:12.925971 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc448d7e-0577-4f62-a701-84c8a5a0873a admin admin] PATH_INFO: `/v3/projects/34866fb3f98042d38c922bf3552c4e69/users/02fd338d362a461a846a454c487bf669/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:12.941510 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc448d7e-0577-4f62-a701-84c8a5a0873a admin admin] RBAC: Authorizing `identity:create_grant(project_id=34866fb3f98042d38c922bf3552c4e69, user_id=02fd338d362a461a846a454c487bf669, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:12.944493 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc448d7e-0577-4f62-a701-84c8a5a0873a admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:12.958785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1069/2137] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:15:12 2022] PUT /identity/v3/projects/34866fb3f98042d38c922bf3552c4e69/users/02fd338d362a461a846a454c487bf669/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:15:12.963799 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d3529-2711-4274-87c8-df49050dc854 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:12.964049 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d3529-2711-4274-87c8-df49050dc854 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:12.964325 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d3529-2711-4274-87c8-df49050dc854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:12.985645 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-ef2d3529-2711-4274-87c8-df49050dc854 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:15:12.986865 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-ef2d3529-2711-4274-87c8-df49050dc854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:13.005996 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:15:13.053554 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1069/2138] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:15:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:15:13.060191 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc448d7e-0577-4f62-a701-84c8a5a0873a admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:13.062351 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-cc448d7e-0577-4f62-a701-84c8a5a0873a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:13.118764 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc448d7e-0577-4f62-a701-84c8a5a0873a admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:13.122607 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78677a7d-9f73-4e7a-830d-f0ee53498195 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:13.123227 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78677a7d-9f73-4e7a-830d-f0ee53498195 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:13.124072 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78677a7d-9f73-4e7a-830d-f0ee53498195 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:13.124340 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78677a7d-9f73-4e7a-830d-f0ee53498195 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:13.133067 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78677a7d-9f73-4e7a-830d-f0ee53498195 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:13.134452 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78677a7d-9f73-4e7a-830d-f0ee53498195 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:13.144772 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1070/2139] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:15:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:15:14.131473 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cae3b9-be81-4592-8267-bf9c78c61b3a None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:14.131958 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cae3b9-be81-4592-8267-bf9c78c61b3a None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:14.132055 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cae3b9-be81-4592-8267-bf9c78c61b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:14.162193 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-e1cae3b9-be81-4592-8267-bf9c78c61b3a None None] MFA Rules not processed for user `02fd338d362a461a846a454c487bf669`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:15:14.163656 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-e1cae3b9-be81-4592-8267-bf9c78c61b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:14.249047 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1070/2140] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:15:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:15:14.267113 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78677a7d-9f73-4e7a-830d-f0ee53498195 service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:14.269309 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-78677a7d-9f73-4e7a-830d-f0ee53498195 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:14.326351 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:15:14.344026 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78677a7d-9f73-4e7a-830d-f0ee53498195 service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:14.347982 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541b921-b96d-4250-881f-20c4769fbfda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:14.348640 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541b921-b96d-4250-881f-20c4769fbfda service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:14.348977 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541b921-b96d-4250-881f-20c4769fbfda service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:14.349221 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541b921-b96d-4250-881f-20c4769fbfda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:14.358653 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3541b921-b96d-4250-881f-20c4769fbfda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:14.360103 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3541b921-b96d-4250-881f-20c4769fbfda service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:14.369277 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1071/2141] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:15:14 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:15:15.835261 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae8b4b-415f-4fbc-98f0-01e71b130060 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:15.836562 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae8b4b-415f-4fbc-98f0-01e71b130060 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:15.837060 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae8b4b-415f-4fbc-98f0-01e71b130060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:15.894865 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-d4ae8b4b-415f-4fbc-98f0-01e71b130060 None None] MFA Rules not processed for user `02fd338d362a461a846a454c487bf669`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:15:15.897171 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-d4ae8b4b-415f-4fbc-98f0-01e71b130060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:15.922540 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1071/2142] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:15:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:15:16.956928 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541b921-b96d-4250-881f-20c4769fbfda service cinder] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:16.958539 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-3541b921-b96d-4250-881f-20c4769fbfda service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:16.972598 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541b921-b96d-4250-881f-20c4769fbfda service cinder] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:16.977063 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9317b-482c-42f0-b1ff-8245c064c8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:16.977683 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9317b-482c-42f0-b1ff-8245c064c8bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:16.977958 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9317b-482c-42f0-b1ff-8245c064c8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:16.978229 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9317b-482c-42f0-b1ff-8245c064c8bb admin admin] PATH_INFO: `/v3/users/740332fdeffc4389bb92478ff737bcd2` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:16.979367 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb9317b-482c-42f0-b1ff-8245c064c8bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=740332fdeffc4389bb92478ff737bcd2)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:16.981263 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb9317b-482c-42f0-b1ff-8245c064c8bb admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:17.051403 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2bb9317b-482c-42f0-b1ff-8245c064c8bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '740332fdeffc4389bb92478ff737bcd2'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:17.066111 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2bb9317b-482c-42f0-b1ff-8245c064c8bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '740332fdeffc4389bb92478ff737bcd2'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:17.068290 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2bb9317b-482c-42f0-b1ff-8245c064c8bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '740332fdeffc4389bb92478ff737bcd2'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:17.069797 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2bb9317b-482c-42f0-b1ff-8245c064c8bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '740332fdeffc4389bb92478ff737bcd2'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:17.087805 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2bb9317b-482c-42f0-b1ff-8245c064c8bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '740332fdeffc4389bb92478ff737bcd2'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:17.116949 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1072/2143] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:15:16 2022] DELETE /identity/v3/users/740332fdeffc4389bb92478ff737bcd2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:15:17.121417 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d918f34-7b77-4e3c-9651-f172d91c4902 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:17.122170 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d918f34-7b77-4e3c-9651-f172d91c4902 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:17.122452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d918f34-7b77-4e3c-9651-f172d91c4902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:17.144283 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-6d918f34-7b77-4e3c-9651-f172d91c4902 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:15:17.145556 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-6d918f34-7b77-4e3c-9651-f172d91c4902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:17.215188 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1072/2144] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:15:17.226094 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9317b-482c-42f0-b1ff-8245c064c8bb admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:17.227607 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-2bb9317b-482c-42f0-b1ff-8245c064c8bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:17.286127 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9317b-482c-42f0-b1ff-8245c064c8bb admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:17.289801 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62381c46-30c5-484c-8326-ab2d329a3587 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:17.290429 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62381c46-30c5-484c-8326-ab2d329a3587 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:17.290702 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62381c46-30c5-484c-8326-ab2d329a3587 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:17.291000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62381c46-30c5-484c-8326-ab2d329a3587 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:17.300428 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62381c46-30c5-484c-8326-ab2d329a3587 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:17.301827 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62381c46-30c5-484c-8326-ab2d329a3587 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:17.312015 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1073/2145] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:15:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:15:17.590402 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d918f34-7b77-4e3c-9651-f172d91c4902 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:17.592588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-6d918f34-7b77-4e3c-9651-f172d91c4902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:17.612726 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d918f34-7b77-4e3c-9651-f172d91c4902 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:17.638876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c3a6a-d1b4-46ab-ac4c-46534ef97f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:17.638876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c3a6a-d1b4-46ab-ac4c-46534ef97f38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:17.638876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c3a6a-d1b4-46ab-ac4c-46534ef97f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:17.638876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c3a6a-d1b4-46ab-ac4c-46534ef97f38 admin admin] PATH_INFO: `/v3/projects/2c7c5afced6545c0b45bb7842169d9ba` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:17.648853 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c3a6a-d1b4-46ab-ac4c-46534ef97f38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c7c5afced6545c0b45bb7842169d9ba)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:17.651240 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c3a6a-d1b4-46ab-ac4c-46534ef97f38 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:17.710548 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:15:17.718064 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-9a0c3a6a-d1b4-46ab-ac4c-46534ef97f38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c7c5afced6545c0b45bb7842169d9ba', 'request_id': 'req-9a0c3a6a-d1b4-46ab-ac4c-46534ef97f38'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:17.723827 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-9a0c3a6a-d1b4-46ab-ac4c-46534ef97f38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c7c5afced6545c0b45bb7842169d9ba', 'request_id': 'req-9a0c3a6a-d1b4-46ab-ac4c-46534ef97f38'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:17.732657 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-9a0c3a6a-d1b4-46ab-ac4c-46534ef97f38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c7c5afced6545c0b45bb7842169d9ba', 'request_id': 'req-9a0c3a6a-d1b4-46ab-ac4c-46534ef97f38'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:17.745220 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-9a0c3a6a-d1b4-46ab-ac4c-46534ef97f38 admin admin] The token cache is being invalidate because project 2c7c5afced6545c0b45bb7842169d9ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:15:17.745579 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-9a0c3a6a-d1b4-46ab-ac4c-46534ef97f38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:17.746304 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-9a0c3a6a-d1b4-46ab-ac4c-46534ef97f38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:17.747856 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1073/2146] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:15:17 2022] DELETE /identity/v3/projects/2c7c5afced6545c0b45bb7842169d9ba => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:15:23.584515 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62381c46-30c5-484c-8326-ab2d329a3587 service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:23.586670 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-62381c46-30c5-484c-8326-ab2d329a3587 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:23.619606 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62381c46-30c5-484c-8326-ab2d329a3587 service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:23.682795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:15:23.690694 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1ed986-639e-4da8-b201-cd492c4b3f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:23.691388 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ed986-639e-4da8-b201-cd492c4b3f13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:23.691850 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ed986-639e-4da8-b201-cd492c4b3f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:23.692200 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ed986-639e-4da8-b201-cd492c4b3f13 admin admin] PATH_INFO: `/v3/users/6437da5191e5497babe94df114c09f3f` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:23.693598 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1ed986-639e-4da8-b201-cd492c4b3f13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6437da5191e5497babe94df114c09f3f)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:23.696025 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1ed986-639e-4da8-b201-cd492c4b3f13 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:23.753975 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-fe1ed986-639e-4da8-b201-cd492c4b3f13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6437da5191e5497babe94df114c09f3f'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:23.761863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-fe1ed986-639e-4da8-b201-cd492c4b3f13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6437da5191e5497babe94df114c09f3f'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:23.762733 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-fe1ed986-639e-4da8-b201-cd492c4b3f13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6437da5191e5497babe94df114c09f3f'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:23.763384 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-fe1ed986-639e-4da8-b201-cd492c4b3f13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6437da5191e5497babe94df114c09f3f'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:23.771221 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-fe1ed986-639e-4da8-b201-cd492c4b3f13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6437da5191e5497babe94df114c09f3f'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:23.789363 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1074/2147] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:15:23 2022] DELETE /identity/v3/users/6437da5191e5497babe94df114c09f3f => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:15:23.796758 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc27776-8d42-45df-8f49-94e887f11d3f None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:23.797215 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc27776-8d42-45df-8f49-94e887f11d3f None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:23.797331 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc27776-8d42-45df-8f49-94e887f11d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:23.821217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-4fc27776-8d42-45df-8f49-94e887f11d3f None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:15:23.822488 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-4fc27776-8d42-45df-8f49-94e887f11d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:23.889195 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1074/2148] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:15:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:15:23.899817 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1ed986-639e-4da8-b201-cd492c4b3f13 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:23.901270 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-fe1ed986-639e-4da8-b201-cd492c4b3f13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:23.962734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1ed986-639e-4da8-b201-cd492c4b3f13 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:23.966446 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd0c03a-de5c-4b6e-8498-51d070d44062 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:23.967634 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0c03a-de5c-4b6e-8498-51d070d44062 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:23.967916 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0c03a-de5c-4b6e-8498-51d070d44062 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:23.968189 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0c03a-de5c-4b6e-8498-51d070d44062 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:23.977387 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd0c03a-de5c-4b6e-8498-51d070d44062 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:23.978803 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd0c03a-de5c-4b6e-8498-51d070d44062 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:23.989077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1075/2149] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:15:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:15:24.289800 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc27776-8d42-45df-8f49-94e887f11d3f None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:24.291455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-4fc27776-8d42-45df-8f49-94e887f11d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:24.305453 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc27776-8d42-45df-8f49-94e887f11d3f None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:24.310179 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf119c-5c3c-4304-b207-650cdb4853a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:24.311360 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf119c-5c3c-4304-b207-650cdb4853a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:24.311635 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf119c-5c3c-4304-b207-650cdb4853a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:24.311905 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf119c-5c3c-4304-b207-650cdb4853a3 admin admin] PATH_INFO: `/v3/projects/06a6d09a62e84a01a95c65a743fc91a0` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:24.313022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdf119c-5c3c-4304-b207-650cdb4853a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06a6d09a62e84a01a95c65a743fc91a0)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:24.315325 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdf119c-5c3c-4304-b207-650cdb4853a3 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:24.370634 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-dbdf119c-5c3c-4304-b207-650cdb4853a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06a6d09a62e84a01a95c65a743fc91a0', 'request_id': 'req-dbdf119c-5c3c-4304-b207-650cdb4853a3'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:24.377425 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-dbdf119c-5c3c-4304-b207-650cdb4853a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06a6d09a62e84a01a95c65a743fc91a0', 'request_id': 'req-dbdf119c-5c3c-4304-b207-650cdb4853a3'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:24.385942 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-dbdf119c-5c3c-4304-b207-650cdb4853a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06a6d09a62e84a01a95c65a743fc91a0', 'request_id': 'req-dbdf119c-5c3c-4304-b207-650cdb4853a3'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:24.398944 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-dbdf119c-5c3c-4304-b207-650cdb4853a3 admin admin] The token cache is being invalidate because project 06a6d09a62e84a01a95c65a743fc91a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:15:24.398944 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-dbdf119c-5c3c-4304-b207-650cdb4853a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:24.399690 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-dbdf119c-5c3c-4304-b207-650cdb4853a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:24.404173 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1075/2150] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:15:24 2022] DELETE /identity/v3/projects/06a6d09a62e84a01a95c65a743fc91a0 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:15:24.405357 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd0c03a-de5c-4b6e-8498-51d070d44062 service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:24.420529 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd0c03a-de5c-4b6e-8498-51d070d44062 service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:24.523484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b2b71c-f99a-4ed3-9ed7-5ca437deb51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:24.524320 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2b71c-f99a-4ed3-9ed7-5ca437deb51c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:24.524893 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2b71c-f99a-4ed3-9ed7-5ca437deb51c admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:24.525468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2b71c-f99a-4ed3-9ed7-5ca437deb51c admin admin] PATH_INFO: `/v3/users/0ae8addce37444f3a629aa059a3efd9a` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:24.527078 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b2b71c-f99a-4ed3-9ed7-5ca437deb51c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ae8addce37444f3a629aa059a3efd9a)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:24.529818 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b2b71c-f99a-4ed3-9ed7-5ca437deb51c admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:24.557317 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:15:24.605003 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-03b2b71c-f99a-4ed3-9ed7-5ca437deb51c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ae8addce37444f3a629aa059a3efd9a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:24.616660 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-03b2b71c-f99a-4ed3-9ed7-5ca437deb51c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ae8addce37444f3a629aa059a3efd9a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:24.617358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-03b2b71c-f99a-4ed3-9ed7-5ca437deb51c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ae8addce37444f3a629aa059a3efd9a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:24.618043 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-03b2b71c-f99a-4ed3-9ed7-5ca437deb51c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ae8addce37444f3a629aa059a3efd9a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:24.629712 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-03b2b71c-f99a-4ed3-9ed7-5ca437deb51c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ae8addce37444f3a629aa059a3efd9a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:24.657044 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1076/2151] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:15:24 2022] DELETE /identity/v3/users/0ae8addce37444f3a629aa059a3efd9a => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:15:25.071426 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf119c-5c3c-4304-b207-650cdb4853a3 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:25.072885 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-dbdf119c-5c3c-4304-b207-650cdb4853a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:25.097558 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf119c-5c3c-4304-b207-650cdb4853a3 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:25.158431 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:15:25.169369 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3cf2ea-ad46-4b9f-bdd6-36bdb991aad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:25.170260 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3cf2ea-ad46-4b9f-bdd6-36bdb991aad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:25.170603 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3cf2ea-ad46-4b9f-bdd6-36bdb991aad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:25.171089 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3cf2ea-ad46-4b9f-bdd6-36bdb991aad8 admin admin] PATH_INFO: `/v3/projects/a3ef98e32a0941948b911bea165a4ecc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:25.172551 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3cf2ea-ad46-4b9f-bdd6-36bdb991aad8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3ef98e32a0941948b911bea165a4ecc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:25.174532 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3cf2ea-ad46-4b9f-bdd6-36bdb991aad8 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:25.239357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-ae3cf2ea-ad46-4b9f-bdd6-36bdb991aad8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3ef98e32a0941948b911bea165a4ecc', 'request_id': 'req-ae3cf2ea-ad46-4b9f-bdd6-36bdb991aad8'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:25.248016 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-ae3cf2ea-ad46-4b9f-bdd6-36bdb991aad8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3ef98e32a0941948b911bea165a4ecc', 'request_id': 'req-ae3cf2ea-ad46-4b9f-bdd6-36bdb991aad8'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:25.257661 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-ae3cf2ea-ad46-4b9f-bdd6-36bdb991aad8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3ef98e32a0941948b911bea165a4ecc', 'request_id': 'req-ae3cf2ea-ad46-4b9f-bdd6-36bdb991aad8'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:25.273154 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-ae3cf2ea-ad46-4b9f-bdd6-36bdb991aad8 admin admin] The token cache is being invalidate because project a3ef98e32a0941948b911bea165a4ecc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:15:25.273458 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-ae3cf2ea-ad46-4b9f-bdd6-36bdb991aad8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:25.274112 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-ae3cf2ea-ad46-4b9f-bdd6-36bdb991aad8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:25.275779 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1076/2152] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:15:25 2022] DELETE /identity/v3/projects/a3ef98e32a0941948b911bea165a4ecc => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:15:25.281135 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56093fb-2c71-4203-a6bf-080ea063e191 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:25.281418 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56093fb-2c71-4203-a6bf-080ea063e191 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:25.281687 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56093fb-2c71-4203-a6bf-080ea063e191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:25.302449 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-e56093fb-2c71-4203-a6bf-080ea063e191 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:15:25.303731 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e56093fb-2c71-4203-a6bf-080ea063e191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:25.388523 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1077/2153] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:15:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:15:25.396811 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec136fa1-ba17-4169-8a35-6923236bbd5b None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:25.397309 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec136fa1-ba17-4169-8a35-6923236bbd5b None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:25.397763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec136fa1-ba17-4169-8a35-6923236bbd5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:25.424230 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-ec136fa1-ba17-4169-8a35-6923236bbd5b None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:15:25.425444 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-ec136fa1-ba17-4169-8a35-6923236bbd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:25.434189 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1077/2154] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:15:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:15:25.439012 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56093fb-2c71-4203-a6bf-080ea063e191 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:25.449350 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56093fb-2c71-4203-a6bf-080ea063e191 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:25.454565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedca7c9-2760-4f49-a7d3-c3e2d82e9f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:25.455211 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedca7c9-2760-4f49-a7d3-c3e2d82e9f42 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:25.455490 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedca7c9-2760-4f49-a7d3-c3e2d82e9f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:25.455759 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedca7c9-2760-4f49-a7d3-c3e2d82e9f42 admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:25.456403 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedca7c9-2760-4f49-a7d3-c3e2d82e9f42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:15:25.457003 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedca7c9-2760-4f49-a7d3-c3e2d82e9f42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:25.459986 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedca7c9-2760-4f49-a7d3-c3e2d82e9f42 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:25.493776 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1078/2155] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:15:25.495638 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec136fa1-ba17-4169-8a35-6923236bbd5b None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:25.507692 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec136fa1-ba17-4169-8a35-6923236bbd5b None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:25.513871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd145a-9300-4781-bbb8-462ca004cb2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:25.514668 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd145a-9300-4781-bbb8-462ca004cb2d admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:25.523052 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd145a-9300-4781-bbb8-462ca004cb2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:25.523052 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd145a-9300-4781-bbb8-462ca004cb2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:25.523052 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd145a-9300-4781-bbb8-462ca004cb2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:25.523052 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd145a-9300-4781-bbb8-462ca004cb2d admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:25.564315 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1078/2156] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:15:25 2022] POST /identity/v3/projects => generated 365 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:15:25.568737 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedca7c9-2760-4f49-a7d3-c3e2d82e9f42 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:25.580610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedca7c9-2760-4f49-a7d3-c3e2d82e9f42 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:25.660897 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba9092-caa2-4917-9efe-304df8c6db3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:25.662320 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba9092-caa2-4917-9efe-304df8c6db3d admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:25.662754 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba9092-caa2-4917-9efe-304df8c6db3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:25.663263 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba9092-caa2-4917-9efe-304df8c6db3d admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:25.664059 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ba9092-caa2-4917-9efe-304df8c6db3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:25.666069 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ba9092-caa2-4917-9efe-304df8c6db3d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:25.696674 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1079/2157] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:15:25 2022] POST /identity/v3/users => generated 283 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:15:25.702075 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd145a-9300-4781-bbb8-462ca004cb2d admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:25.714951 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd145a-9300-4781-bbb8-462ca004cb2d admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:25.722075 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be5081-f8fa-4ed3-9c24-a06da07154df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:25.722811 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be5081-f8fa-4ed3-9c24-a06da07154df admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:25.723221 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be5081-f8fa-4ed3-9c24-a06da07154df admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:25.723585 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be5081-f8fa-4ed3-9c24-a06da07154df admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:25.724453 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08be5081-f8fa-4ed3-9c24-a06da07154df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:25.726511 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08be5081-f8fa-4ed3-9c24-a06da07154df admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:25.738876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1079/2158] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:15:25.742610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba9092-caa2-4917-9efe-304df8c6db3d admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:25.753764 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba9092-caa2-4917-9efe-304df8c6db3d admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:25.758728 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323339cf-b72e-435b-a830-72c79a5b4409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:25.759368 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323339cf-b72e-435b-a830-72c79a5b4409 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:25.760264 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323339cf-b72e-435b-a830-72c79a5b4409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:25.760701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323339cf-b72e-435b-a830-72c79a5b4409 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:25.761497 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323339cf-b72e-435b-a830-72c79a5b4409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:25.763265 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323339cf-b72e-435b-a830-72c79a5b4409 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:25.776861 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1080/2159] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:15:25.781466 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be5081-f8fa-4ed3-9c24-a06da07154df admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:25.792791 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be5081-f8fa-4ed3-9c24-a06da07154df admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:25.798174 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9f42b6-cb80-4d57-ace0-606a20d09a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:25.798822 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f42b6-cb80-4d57-ace0-606a20d09a03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:25.799125 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f42b6-cb80-4d57-ace0-606a20d09a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:25.799397 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f42b6-cb80-4d57-ace0-606a20d09a03 admin admin] PATH_INFO: `/v3/projects/b8d3883b0e504452b1d88defc505ea8b/users/bd23b04af7944ef4ad23386eaadb2792/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:25.819476 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9f42b6-cb80-4d57-ace0-606a20d09a03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8d3883b0e504452b1d88defc505ea8b, user_id=bd23b04af7944ef4ad23386eaadb2792, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:25.822704 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9f42b6-cb80-4d57-ace0-606a20d09a03 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:25.839971 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1080/2160] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:15:25 2022] PUT /identity/v3/projects/b8d3883b0e504452b1d88defc505ea8b/users/bd23b04af7944ef4ad23386eaadb2792/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:15:25.852123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe88b706-afcd-41a4-a6b7-f45df63973c9 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:25.852713 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe88b706-afcd-41a4-a6b7-f45df63973c9 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:25.853084 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe88b706-afcd-41a4-a6b7-f45df63973c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:25.862744 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:15:25.884842 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-fe88b706-afcd-41a4-a6b7-f45df63973c9 None None] MFA Rules not processed for user `bd23b04af7944ef4ad23386eaadb2792`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:15:25.886217 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-fe88b706-afcd-41a4-a6b7-f45df63973c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:25.972041 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1081/2161] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:15:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:15:25.976730 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9f42b6-cb80-4d57-ace0-606a20d09a03 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:26.056243 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9f42b6-cb80-4d57-ace0-606a20d09a03 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:26.062189 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201dabb0-0f22-4193-8b40-9cd33895d1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:26.063230 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201dabb0-0f22-4193-8b40-9cd33895d1ca admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:26.063874 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201dabb0-0f22-4193-8b40-9cd33895d1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:26.064209 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201dabb0-0f22-4193-8b40-9cd33895d1ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:26.065109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201dabb0-0f22-4193-8b40-9cd33895d1ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:26.067604 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201dabb0-0f22-4193-8b40-9cd33895d1ca admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:26.105634 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1081/2162] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:15:25 2022] POST /identity/v3/projects => generated 363 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:15:26.112133 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe88b706-afcd-41a4-a6b7-f45df63973c9 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:26.131708 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe88b706-afcd-41a4-a6b7-f45df63973c9 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:26.210287 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82faea6a-bcf0-4e05-8405-8bb5b00085af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:26.210934 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82faea6a-bcf0-4e05-8405-8bb5b00085af admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:26.211267 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82faea6a-bcf0-4e05-8405-8bb5b00085af admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:26.211608 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82faea6a-bcf0-4e05-8405-8bb5b00085af admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:26.212201 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82faea6a-bcf0-4e05-8405-8bb5b00085af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:26.214508 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82faea6a-bcf0-4e05-8405-8bb5b00085af admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:26.246317 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1082/2163] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:15:26 2022] POST /identity/v3/users => generated 288 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:15:26.247626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201dabb0-0f22-4193-8b40-9cd33895d1ca admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:26.255892 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:15:26.260233 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201dabb0-0f22-4193-8b40-9cd33895d1ca admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:26.281062 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ae62b-f04c-434c-910f-2d1a8452bd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:26.282302 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ae62b-f04c-434c-910f-2d1a8452bd74 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:26.282787 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ae62b-f04c-434c-910f-2d1a8452bd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:26.283323 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ae62b-f04c-434c-910f-2d1a8452bd74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:26.284207 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456ae62b-f04c-434c-910f-2d1a8452bd74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:26.297336 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456ae62b-f04c-434c-910f-2d1a8452bd74 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:26.320618 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1082/2164] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:15:26.321837 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82faea6a-bcf0-4e05-8405-8bb5b00085af admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:26.356051 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82faea6a-bcf0-4e05-8405-8bb5b00085af admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:26.361431 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5541408-ac83-468d-83ac-89d3135af60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:26.362273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5541408-ac83-468d-83ac-89d3135af60d admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:26.362623 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5541408-ac83-468d-83ac-89d3135af60d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:26.363122 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5541408-ac83-468d-83ac-89d3135af60d admin admin] PATH_INFO: `/v3/projects/86070fff6b5f47beb060d79547475c31/users/d13eec7cf88641a2ac09bcf313c03c5b/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:26.381665 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5541408-ac83-468d-83ac-89d3135af60d admin admin] RBAC: Authorizing `identity:create_grant(project_id=86070fff6b5f47beb060d79547475c31, user_id=d13eec7cf88641a2ac09bcf313c03c5b, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:26.385156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5541408-ac83-468d-83ac-89d3135af60d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:26.401145 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1083/2165] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:15:26 2022] PUT /identity/v3/projects/86070fff6b5f47beb060d79547475c31/users/d13eec7cf88641a2ac09bcf313c03c5b/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:15:26.405033 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ae62b-f04c-434c-910f-2d1a8452bd74 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:26.416344 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ae62b-f04c-434c-910f-2d1a8452bd74 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:26.473874 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73062c3e-10c9-44a3-9d72-499ce2a7c852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:26.474395 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73062c3e-10c9-44a3-9d72-499ce2a7c852 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:26.474670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73062c3e-10c9-44a3-9d72-499ce2a7c852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:26.474971 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73062c3e-10c9-44a3-9d72-499ce2a7c852 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:26.475602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73062c3e-10c9-44a3-9d72-499ce2a7c852 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:26.477205 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73062c3e-10c9-44a3-9d72-499ce2a7c852 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:26.490014 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1083/2166] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:15:26.491397 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5541408-ac83-468d-83ac-89d3135af60d admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:26.502123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5541408-ac83-468d-83ac-89d3135af60d admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:26.507004 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43a050-bc6d-4301-8c2d-1012fa15f6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:26.507873 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43a050-bc6d-4301-8c2d-1012fa15f6bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:26.508162 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43a050-bc6d-4301-8c2d-1012fa15f6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:26.508439 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43a050-bc6d-4301-8c2d-1012fa15f6bc admin admin] PATH_INFO: `/v3/projects/86070fff6b5f47beb060d79547475c31/users/d13eec7cf88641a2ac09bcf313c03c5b/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:26.510399 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a43a050-bc6d-4301-8c2d-1012fa15f6bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=86070fff6b5f47beb060d79547475c31, user_id=d13eec7cf88641a2ac09bcf313c03c5b, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:26.513080 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a43a050-bc6d-4301-8c2d-1012fa15f6bc admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:26.523919 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1084/2167] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:15:26 2022] PUT /identity/v3/projects/86070fff6b5f47beb060d79547475c31/users/d13eec7cf88641a2ac09bcf313c03c5b/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:15:26.548378 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722d9ff-c3a3-4ae7-a48b-230ab0d0abc0 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:26.548605 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722d9ff-c3a3-4ae7-a48b-230ab0d0abc0 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:26.548957 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722d9ff-c3a3-4ae7-a48b-230ab0d0abc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:26.589352 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-e722d9ff-c3a3-4ae7-a48b-230ab0d0abc0 None None] MFA Rules not processed for user `d13eec7cf88641a2ac09bcf313c03c5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:15:26.590696 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-e722d9ff-c3a3-4ae7-a48b-230ab0d0abc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:26.618297 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d008233-a177-45bd-9683-464bd655a9f5 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:26.618611 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d008233-a177-45bd-9683-464bd655a9f5 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:26.618916 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d008233-a177-45bd-9683-464bd655a9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:26.663107 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-2d008233-a177-45bd-9683-464bd655a9f5 None None] MFA Rules not processed for user `5f2ff5e497b64e49b7ac25a941a06048`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:15:26.663107 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-2d008233-a177-45bd-9683-464bd655a9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:26.676214 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1084/2168] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:15:26 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:15:26.688363 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e722d9ff-c3a3-4ae7-a48b-230ab0d0abc0 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:26.689952 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-e722d9ff-c3a3-4ae7-a48b-230ab0d0abc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:26.710906 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:15:26.726181 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1085/2169] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:15:26 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:15:26.750442 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e722d9ff-c3a3-4ae7-a48b-230ab0d0abc0 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:26.755520 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b64d29-e89f-4ebf-89c7-197c8cac5ef4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:26.756267 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b64d29-e89f-4ebf-89c7-197c8cac5ef4 service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:26.756505 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b64d29-e89f-4ebf-89c7-197c8cac5ef4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:26.756781 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b64d29-e89f-4ebf-89c7-197c8cac5ef4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:26.768485 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b64d29-e89f-4ebf-89c7-197c8cac5ef4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:26.769907 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b64d29-e89f-4ebf-89c7-197c8cac5ef4 service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:26.779605 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1085/2170] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:15:26 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:15:26.862541 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d008233-a177-45bd-9683-464bd655a9f5 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:26.872087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d008233-a177-45bd-9683-464bd655a9f5 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:26.876286 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ba4ff-5a4e-4d6f-84d4-995096a5988b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:26.876878 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ba4ff-5a4e-4d6f-84d4-995096a5988b service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:26.877162 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ba4ff-5a4e-4d6f-84d4-995096a5988b service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:26.877433 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ba4ff-5a4e-4d6f-84d4-995096a5988b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:26.934100 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ba4ff-5a4e-4d6f-84d4-995096a5988b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:26.936739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ba4ff-5a4e-4d6f-84d4-995096a5988b service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:26.945989 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1086/2171] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:15:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:15:29.854263 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b64d29-e89f-4ebf-89c7-197c8cac5ef4 service cinder] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:29.894815 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:15:29.968190 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b64d29-e89f-4ebf-89c7-197c8cac5ef4 service cinder] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:29.974052 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ad609-4721-4cdc-b9b9-f084b80cf20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:29.974705 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ad609-4721-4cdc-b9b9-f084b80cf20b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:29.975055 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ad609-4721-4cdc-b9b9-f084b80cf20b admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:29.975320 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ad609-4721-4cdc-b9b9-f084b80cf20b admin admin] PATH_INFO: `/v3/users/bd23b04af7944ef4ad23386eaadb2792` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:29.976424 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723ad609-4721-4cdc-b9b9-f084b80cf20b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd23b04af7944ef4ad23386eaadb2792)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:29.978364 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723ad609-4721-4cdc-b9b9-f084b80cf20b admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:30.037683 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-723ad609-4721-4cdc-b9b9-f084b80cf20b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd23b04af7944ef4ad23386eaadb2792'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:30.046563 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-723ad609-4721-4cdc-b9b9-f084b80cf20b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd23b04af7944ef4ad23386eaadb2792'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:30.047270 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-723ad609-4721-4cdc-b9b9-f084b80cf20b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd23b04af7944ef4ad23386eaadb2792'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:30.048039 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-723ad609-4721-4cdc-b9b9-f084b80cf20b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd23b04af7944ef4ad23386eaadb2792'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:30.055508 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-723ad609-4721-4cdc-b9b9-f084b80cf20b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd23b04af7944ef4ad23386eaadb2792'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:30.075088 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1086/2172] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:15:29 2022] DELETE /identity/v3/users/bd23b04af7944ef4ad23386eaadb2792 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:15:30.079664 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f868b3-5611-45b3-a86d-1f627ca3c422 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:30.080152 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f868b3-5611-45b3-a86d-1f627ca3c422 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:30.080316 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f868b3-5611-45b3-a86d-1f627ca3c422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:30.101875 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-d6f868b3-5611-45b3-a86d-1f627ca3c422 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:15:30.103205 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d6f868b3-5611-45b3-a86d-1f627ca3c422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:30.201531 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1087/2173] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:15:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:15:30.218871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ad609-4721-4cdc-b9b9-f084b80cf20b admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:30.221103 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-723ad609-4721-4cdc-b9b9-f084b80cf20b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:30.301764 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ad609-4721-4cdc-b9b9-f084b80cf20b admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:30.305612 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2144a1eb-47e3-468a-96a7-aaab44014ef5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:30.306218 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144a1eb-47e3-468a-96a7-aaab44014ef5 service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:30.306494 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144a1eb-47e3-468a-96a7-aaab44014ef5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:30.306765 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144a1eb-47e3-468a-96a7-aaab44014ef5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:30.333318 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2144a1eb-47e3-468a-96a7-aaab44014ef5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:30.335160 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2144a1eb-47e3-468a-96a7-aaab44014ef5 service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:30.362817 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1087/2174] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:15:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:15:31.052634 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f868b3-5611-45b3-a86d-1f627ca3c422 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:31.054136 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d6f868b3-5611-45b3-a86d-1f627ca3c422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:31.082890 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f868b3-5611-45b3-a86d-1f627ca3c422 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:31.082890 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb340e0a-9640-44f4-99f4-2ce1c97f5647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:31.082890 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb340e0a-9640-44f4-99f4-2ce1c97f5647 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:31.082890 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb340e0a-9640-44f4-99f4-2ce1c97f5647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:31.083812 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb340e0a-9640-44f4-99f4-2ce1c97f5647 admin admin] PATH_INFO: `/v3/projects/b8d3883b0e504452b1d88defc505ea8b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:31.084551 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb340e0a-9640-44f4-99f4-2ce1c97f5647 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8d3883b0e504452b1d88defc505ea8b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:31.086517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb340e0a-9640-44f4-99f4-2ce1c97f5647 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:31.137837 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-eb340e0a-9640-44f4-99f4-2ce1c97f5647 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8d3883b0e504452b1d88defc505ea8b', 'request_id': 'req-eb340e0a-9640-44f4-99f4-2ce1c97f5647'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:31.147652 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-eb340e0a-9640-44f4-99f4-2ce1c97f5647 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8d3883b0e504452b1d88defc505ea8b', 'request_id': 'req-eb340e0a-9640-44f4-99f4-2ce1c97f5647'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:31.154638 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-eb340e0a-9640-44f4-99f4-2ce1c97f5647 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8d3883b0e504452b1d88defc505ea8b', 'request_id': 'req-eb340e0a-9640-44f4-99f4-2ce1c97f5647'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:31.166080 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-eb340e0a-9640-44f4-99f4-2ce1c97f5647 admin admin] The token cache is being invalidate because project b8d3883b0e504452b1d88defc505ea8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:15:31.166369 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-eb340e0a-9640-44f4-99f4-2ce1c97f5647 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:31.167116 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-eb340e0a-9640-44f4-99f4-2ce1c97f5647 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:31.168592 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1088/2175] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:15:31 2022] DELETE /identity/v3/projects/b8d3883b0e504452b1d88defc505ea8b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:15:31.172660 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2144a1eb-47e3-468a-96a7-aaab44014ef5 service neutron] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:31.182285 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2144a1eb-47e3-468a-96a7-aaab44014ef5 service neutron] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:31.220267 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:15:31.240750 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb80cb8-8353-4fe1-a38f-8ad07d7b622f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:31.241385 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb80cb8-8353-4fe1-a38f-8ad07d7b622f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:31.241665 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb80cb8-8353-4fe1-a38f-8ad07d7b622f admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:31.241942 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb80cb8-8353-4fe1-a38f-8ad07d7b622f admin admin] PATH_INFO: `/v3/users/d13eec7cf88641a2ac09bcf313c03c5b` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:31.243094 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb80cb8-8353-4fe1-a38f-8ad07d7b622f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d13eec7cf88641a2ac09bcf313c03c5b)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:31.245010 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb80cb8-8353-4fe1-a38f-8ad07d7b622f admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:31.307616 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-ffb80cb8-8353-4fe1-a38f-8ad07d7b622f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd13eec7cf88641a2ac09bcf313c03c5b'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:31.317844 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-ffb80cb8-8353-4fe1-a38f-8ad07d7b622f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd13eec7cf88641a2ac09bcf313c03c5b'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:31.318750 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-ffb80cb8-8353-4fe1-a38f-8ad07d7b622f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd13eec7cf88641a2ac09bcf313c03c5b'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:31.319463 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-ffb80cb8-8353-4fe1-a38f-8ad07d7b622f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd13eec7cf88641a2ac09bcf313c03c5b'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:31.328432 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-ffb80cb8-8353-4fe1-a38f-8ad07d7b622f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd13eec7cf88641a2ac09bcf313c03c5b'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:31.354283 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1088/2176] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:15:31 2022] DELETE /identity/v3/users/d13eec7cf88641a2ac09bcf313c03c5b => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:15:31.752156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb340e0a-9640-44f4-99f4-2ce1c97f5647 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:31.753876 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-eb340e0a-9640-44f4-99f4-2ce1c97f5647 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:31.767705 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb340e0a-9640-44f4-99f4-2ce1c97f5647 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:31.778126 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:15:31.846338 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d2c81-3393-4b21-a729-c2eabebb0058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:31.847088 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d2c81-3393-4b21-a729-c2eabebb0058 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:31.847494 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d2c81-3393-4b21-a729-c2eabebb0058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:31.847728 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d2c81-3393-4b21-a729-c2eabebb0058 admin admin] PATH_INFO: `/v3/projects/86070fff6b5f47beb060d79547475c31` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:31.848866 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7d2c81-3393-4b21-a729-c2eabebb0058 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86070fff6b5f47beb060d79547475c31)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:31.850760 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7d2c81-3393-4b21-a729-c2eabebb0058 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:31.899234 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-5f7d2c81-3393-4b21-a729-c2eabebb0058 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86070fff6b5f47beb060d79547475c31', 'request_id': 'req-5f7d2c81-3393-4b21-a729-c2eabebb0058'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:31.907800 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-5f7d2c81-3393-4b21-a729-c2eabebb0058 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86070fff6b5f47beb060d79547475c31', 'request_id': 'req-5f7d2c81-3393-4b21-a729-c2eabebb0058'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:31.915539 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-5f7d2c81-3393-4b21-a729-c2eabebb0058 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86070fff6b5f47beb060d79547475c31', 'request_id': 'req-5f7d2c81-3393-4b21-a729-c2eabebb0058'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:31.928049 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-5f7d2c81-3393-4b21-a729-c2eabebb0058 admin admin] The token cache is being invalidate because project 86070fff6b5f47beb060d79547475c31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:15:31.928329 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-5f7d2c81-3393-4b21-a729-c2eabebb0058 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:31.929038 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-5f7d2c81-3393-4b21-a729-c2eabebb0058 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:31.930809 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1089/2177] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:15:31 2022] DELETE /identity/v3/projects/86070fff6b5f47beb060d79547475c31 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:15:31.937064 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd68064-fbe7-469f-bbb3-ce1608dc35c1 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:31.937461 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd68064-fbe7-469f-bbb3-ce1608dc35c1 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:31.937829 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd68064-fbe7-469f-bbb3-ce1608dc35c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:31.958472 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-5cd68064-fbe7-469f-bbb3-ce1608dc35c1 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:15:31.959774 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-5cd68064-fbe7-469f-bbb3-ce1608dc35c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:32.018064 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1089/2178] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:15:32.026796 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26366ddd-aee5-4825-8e79-126e783dd88e None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:32.027113 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26366ddd-aee5-4825-8e79-126e783dd88e None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:32.027386 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26366ddd-aee5-4825-8e79-126e783dd88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:32.047627 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-26366ddd-aee5-4825-8e79-126e783dd88e None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:15:32.048832 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-26366ddd-aee5-4825-8e79-126e783dd88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:32.056793 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1090/2179] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:15:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:15:32.062455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd68064-fbe7-469f-bbb3-ce1608dc35c1 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:32.076563 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd68064-fbe7-469f-bbb3-ce1608dc35c1 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:32.081877 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ac674-3e1b-4493-8738-5cc10fc2db98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:32.082565 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ac674-3e1b-4493-8738-5cc10fc2db98 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:32.082944 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ac674-3e1b-4493-8738-5cc10fc2db98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:32.083288 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ac674-3e1b-4493-8738-5cc10fc2db98 admin admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:32.084003 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ac674-3e1b-4493-8738-5cc10fc2db98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:15:32.084373 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ac674-3e1b-4493-8738-5cc10fc2db98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:32.086020 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ac674-3e1b-4493-8738-5cc10fc2db98 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:32.099668 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1090/2180] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:15:32.103570 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26366ddd-aee5-4825-8e79-126e783dd88e None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:32.112703 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26366ddd-aee5-4825-8e79-126e783dd88e None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:32.117294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61642d7a-dfe1-4bc9-9397-7badb66c0101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:32.117901 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61642d7a-dfe1-4bc9-9397-7badb66c0101 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:32.118174 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61642d7a-dfe1-4bc9-9397-7badb66c0101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:32.118550 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61642d7a-dfe1-4bc9-9397-7badb66c0101 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:32.119101 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61642d7a-dfe1-4bc9-9397-7badb66c0101 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:32.120971 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61642d7a-dfe1-4bc9-9397-7badb66c0101 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:32.160425 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1091/2181] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:15:32 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:15:32.161783 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ac674-3e1b-4493-8738-5cc10fc2db98 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:32.172326 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ac674-3e1b-4493-8738-5cc10fc2db98 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:32.226303 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:15:32.248465 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd124bbe-a599-49f7-883f-c25027139719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:32.249264 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd124bbe-a599-49f7-883f-c25027139719 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:32.249539 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd124bbe-a599-49f7-883f-c25027139719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:32.249810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd124bbe-a599-49f7-883f-c25027139719 admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:32.250526 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd124bbe-a599-49f7-883f-c25027139719 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:32.253070 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd124bbe-a599-49f7-883f-c25027139719 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:32.285353 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1091/2182] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:15:32 2022] POST /identity/v3/users => generated 292 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:15:32.285943 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61642d7a-dfe1-4bc9-9397-7badb66c0101 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:32.299986 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61642d7a-dfe1-4bc9-9397-7badb66c0101 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:32.305092 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6b112-fdc0-4dda-9044-103f54c015e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:32.305702 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6b112-fdc0-4dda-9044-103f54c015e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:32.305976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6b112-fdc0-4dda-9044-103f54c015e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:32.306246 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6b112-fdc0-4dda-9044-103f54c015e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:32.307057 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f6b112-fdc0-4dda-9044-103f54c015e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:32.308631 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f6b112-fdc0-4dda-9044-103f54c015e7 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:32.319702 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1092/2183] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:15:32.323487 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd124bbe-a599-49f7-883f-c25027139719 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:32.334130 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd124bbe-a599-49f7-883f-c25027139719 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:32.339564 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a6b7d-b511-4cb1-9938-1958de2de4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:32.340167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a6b7d-b511-4cb1-9938-1958de2de4b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:32.340509 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a6b7d-b511-4cb1-9938-1958de2de4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:32.340776 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a6b7d-b511-4cb1-9938-1958de2de4b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:32.341415 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130a6b7d-b511-4cb1-9938-1958de2de4b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:32.343719 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130a6b7d-b511-4cb1-9938-1958de2de4b6 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:32.356402 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1092/2184] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:15:32.360389 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6b112-fdc0-4dda-9044-103f54c015e7 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:32.376111 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6b112-fdc0-4dda-9044-103f54c015e7 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:32.384208 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c36f88-0833-416a-8be2-e9f220888d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:32.385333 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c36f88-0833-416a-8be2-e9f220888d9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:32.385743 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c36f88-0833-416a-8be2-e9f220888d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:32.386119 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c36f88-0833-416a-8be2-e9f220888d9d admin admin] PATH_INFO: `/v3/projects/cc365cc944a3432d9758373cdd8453e6/users/19862e2c66094c3db3721360161f21b8/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:32.411352 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c36f88-0833-416a-8be2-e9f220888d9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc365cc944a3432d9758373cdd8453e6, user_id=19862e2c66094c3db3721360161f21b8, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:32.427148 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c36f88-0833-416a-8be2-e9f220888d9d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:32.431244 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1093/2185] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:15:32 2022] PUT /identity/v3/projects/cc365cc944a3432d9758373cdd8453e6/users/19862e2c66094c3db3721360161f21b8/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:15:32.440093 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa62c5-cc6d-41e3-902c-0ef245b74b79 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:32.440382 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa62c5-cc6d-41e3-902c-0ef245b74b79 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:32.440734 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa62c5-cc6d-41e3-902c-0ef245b74b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:32.470880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-7efa62c5-cc6d-41e3-902c-0ef245b74b79 None None] MFA Rules not processed for user `19862e2c66094c3db3721360161f21b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:15:32.472279 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-7efa62c5-cc6d-41e3-902c-0ef245b74b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:32.550522 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1093/2186] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:15:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:15:32.562048 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c36f88-0833-416a-8be2-e9f220888d9d admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:32.564924 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-20c36f88-0833-416a-8be2-e9f220888d9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:32.616618 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:15:32.637095 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c36f88-0833-416a-8be2-e9f220888d9d admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:32.642403 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2229d2ef-917c-45f5-b805-c8cee82792ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:32.643385 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229d2ef-917c-45f5-b805-c8cee82792ed service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:32.643977 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229d2ef-917c-45f5-b805-c8cee82792ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:32.644310 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229d2ef-917c-45f5-b805-c8cee82792ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:32.659847 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2229d2ef-917c-45f5-b805-c8cee82792ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:32.661403 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2229d2ef-917c-45f5-b805-c8cee82792ed service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:32.672713 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1094/2187] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:15:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:15:32.694049 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa62c5-cc6d-41e3-902c-0ef245b74b79 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:32.763873 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa62c5-cc6d-41e3-902c-0ef245b74b79 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:32.769047 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868a5ed-e27d-4ff3-af93-c894e6951bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:32.769681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868a5ed-e27d-4ff3-af93-c894e6951bac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:32.769955 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868a5ed-e27d-4ff3-af93-c894e6951bac admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:32.770225 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868a5ed-e27d-4ff3-af93-c894e6951bac admin admin] PATH_INFO: `/v3/users/19862e2c66094c3db3721360161f21b8` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:32.771393 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a868a5ed-e27d-4ff3-af93-c894e6951bac admin admin] RBAC: Authorizing `identity:delete_user(user_id=19862e2c66094c3db3721360161f21b8)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:32.773392 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a868a5ed-e27d-4ff3-af93-c894e6951bac admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:32.827050 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-a868a5ed-e27d-4ff3-af93-c894e6951bac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19862e2c66094c3db3721360161f21b8'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:32.841476 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-a868a5ed-e27d-4ff3-af93-c894e6951bac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19862e2c66094c3db3721360161f21b8'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:32.842139 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-a868a5ed-e27d-4ff3-af93-c894e6951bac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19862e2c66094c3db3721360161f21b8'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:32.842868 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-a868a5ed-e27d-4ff3-af93-c894e6951bac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19862e2c66094c3db3721360161f21b8'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:32.845132 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:15:32.852664 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-a868a5ed-e27d-4ff3-af93-c894e6951bac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19862e2c66094c3db3721360161f21b8'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:32.871143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1094/2188] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:15:32 2022] DELETE /identity/v3/users/19862e2c66094c3db3721360161f21b8 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:15:32.876685 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6424f-f49e-4040-b499-f9569e7beee8 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:32.876970 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6424f-f49e-4040-b499-f9569e7beee8 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:32.877243 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6424f-f49e-4040-b499-f9569e7beee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:32.897503 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-f2a6424f-f49e-4040-b499-f9569e7beee8 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:15:32.898648 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f2a6424f-f49e-4040-b499-f9569e7beee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:32.960068 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1095/2189] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:15:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:15:32.971784 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868a5ed-e27d-4ff3-af93-c894e6951bac admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:32.973257 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-a868a5ed-e27d-4ff3-af93-c894e6951bac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:33.035771 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868a5ed-e27d-4ff3-af93-c894e6951bac admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:33.039370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af79707-43c6-4aef-bc28-6323042083a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:33.039975 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af79707-43c6-4aef-bc28-6323042083a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:33.040281 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af79707-43c6-4aef-bc28-6323042083a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:33.040553 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af79707-43c6-4aef-bc28-6323042083a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:33.050507 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af79707-43c6-4aef-bc28-6323042083a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:33.052032 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af79707-43c6-4aef-bc28-6323042083a3 service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:33.064300 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1095/2190] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:15:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:15:33.588646 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a6424f-f49e-4040-b499-f9569e7beee8 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:33.590627 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f2a6424f-f49e-4040-b499-f9569e7beee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:33.607955 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a6424f-f49e-4040-b499-f9569e7beee8 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:33.613699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff4bde-3294-4fe3-b63e-b71969970006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:33.614491 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff4bde-3294-4fe3-b63e-b71969970006 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:33.614856 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff4bde-3294-4fe3-b63e-b71969970006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:33.615237 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff4bde-3294-4fe3-b63e-b71969970006 admin admin] PATH_INFO: `/v3/projects/cc365cc944a3432d9758373cdd8453e6` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:33.616502 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ff4bde-3294-4fe3-b63e-b71969970006 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc365cc944a3432d9758373cdd8453e6)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:33.618593 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ff4bde-3294-4fe3-b63e-b71969970006 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:33.676334 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-53ff4bde-3294-4fe3-b63e-b71969970006 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc365cc944a3432d9758373cdd8453e6', 'request_id': 'req-53ff4bde-3294-4fe3-b63e-b71969970006'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:33.686491 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-53ff4bde-3294-4fe3-b63e-b71969970006 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc365cc944a3432d9758373cdd8453e6', 'request_id': 'req-53ff4bde-3294-4fe3-b63e-b71969970006'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:33.693076 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-53ff4bde-3294-4fe3-b63e-b71969970006 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc365cc944a3432d9758373cdd8453e6', 'request_id': 'req-53ff4bde-3294-4fe3-b63e-b71969970006'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:33.706512 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-53ff4bde-3294-4fe3-b63e-b71969970006 admin admin] The token cache is being invalidate because project cc365cc944a3432d9758373cdd8453e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:15:33.706801 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-53ff4bde-3294-4fe3-b63e-b71969970006 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:33.707477 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-53ff4bde-3294-4fe3-b63e-b71969970006 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:33.709168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1096/2191] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:15:33 2022] DELETE /identity/v3/projects/cc365cc944a3432d9758373cdd8453e6 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:15:33.714041 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dad412-7ef4-48b5-8167-6474af7ef99d None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:33.714476 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dad412-7ef4-48b5-8167-6474af7ef99d None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:33.714641 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dad412-7ef4-48b5-8167-6474af7ef99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:33.756056 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d012b-4593-46f6-98c4-43e6520c58cd None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:33.756341 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d012b-4593-46f6-98c4-43e6520c58cd None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:33.756614 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d012b-4593-46f6-98c4-43e6520c58cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:33.763114 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-f3dad412-7ef4-48b5-8167-6474af7ef99d None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:15:33.765167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-f3dad412-7ef4-48b5-8167-6474af7ef99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:33.778269 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-186d012b-4593-46f6-98c4-43e6520c58cd None None] MFA Rules not processed for user `fff8e93029f447bb8294d9b516eb407f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:15:33.779723 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-186d012b-4593-46f6-98c4-43e6520c58cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:33.853328 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1096/2192] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:15:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:15:33.858404 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1097/2193] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:15:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:15:33.868275 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e493fb-2803-4de8-ae32-ae77cc4fc971 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:33.868551 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e493fb-2803-4de8-ae32-ae77cc4fc971 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:33.868824 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e493fb-2803-4de8-ae32-ae77cc4fc971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:33.893395 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-d3e493fb-2803-4de8-ae32-ae77cc4fc971 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:15:33.895509 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-d3e493fb-2803-4de8-ae32-ae77cc4fc971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:33.916051 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1097/2194] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:15:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:15:33.918648 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d012b-4593-46f6-98c4-43e6520c58cd None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:33.931158 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d012b-4593-46f6-98c4-43e6520c58cd None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:33.937203 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e878c-a90c-4312-8afd-114abc03cc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:33.937812 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e878c-a90c-4312-8afd-114abc03cc7d admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:33.938146 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e878c-a90c-4312-8afd-114abc03cc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:33.938683 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e878c-a90c-4312-8afd-114abc03cc7d admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:33.939456 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e878c-a90c-4312-8afd-114abc03cc7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:15:33.939743 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e878c-a90c-4312-8afd-114abc03cc7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:33.946504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e878c-a90c-4312-8afd-114abc03cc7d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:33.954746 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:15:33.965447 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1098/2195] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:15:33.966359 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e493fb-2803-4de8-ae32-ae77cc4fc971 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:33.976801 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e493fb-2803-4de8-ae32-ae77cc4fc971 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:33.982083 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcba54e-28c2-44ac-b1c6-2feb237792cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:33.982487 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcba54e-28c2-44ac-b1c6-2feb237792cf admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:33.982759 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcba54e-28c2-44ac-b1c6-2feb237792cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:33.983058 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcba54e-28c2-44ac-b1c6-2feb237792cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:33.984313 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcba54e-28c2-44ac-b1c6-2feb237792cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:33.986210 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcba54e-28c2-44ac-b1c6-2feb237792cf admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:34.023203 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1098/2196] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:15:33 2022] POST /identity/v3/projects => generated 391 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:15:34.027564 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e878c-a90c-4312-8afd-114abc03cc7d admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:34.039348 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e878c-a90c-4312-8afd-114abc03cc7d admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:34.115398 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e1a1b8-ee7d-49bb-9a94-a216b1d70014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:34.116037 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1a1b8-ee7d-49bb-9a94-a216b1d70014 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:34.116311 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1a1b8-ee7d-49bb-9a94-a216b1d70014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:34.116581 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1a1b8-ee7d-49bb-9a94-a216b1d70014 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:34.117223 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e1a1b8-ee7d-49bb-9a94-a216b1d70014 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:34.119314 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e1a1b8-ee7d-49bb-9a94-a216b1d70014 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:34.142641 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1099/2197] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:15:34 2022] POST /identity/v3/users => generated 296 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:15:34.147086 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcba54e-28c2-44ac-b1c6-2feb237792cf admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:34.158224 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcba54e-28c2-44ac-b1c6-2feb237792cf admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:34.162876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92789bfb-8cfe-4995-ab07-3495c6d8a00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:34.163490 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92789bfb-8cfe-4995-ab07-3495c6d8a00d admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:34.163766 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92789bfb-8cfe-4995-ab07-3495c6d8a00d admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:34.164038 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92789bfb-8cfe-4995-ab07-3495c6d8a00d admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:34.164681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92789bfb-8cfe-4995-ab07-3495c6d8a00d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:34.166271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92789bfb-8cfe-4995-ab07-3495c6d8a00d admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:34.177899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1099/2198] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:15:34.181771 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e1a1b8-ee7d-49bb-9a94-a216b1d70014 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:34.192593 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e1a1b8-ee7d-49bb-9a94-a216b1d70014 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:34.198660 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc953df-f47c-4ad9-86af-0756da1bd8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:34.199308 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc953df-f47c-4ad9-86af-0756da1bd8ae admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:34.199584 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc953df-f47c-4ad9-86af-0756da1bd8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:34.200588 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc953df-f47c-4ad9-86af-0756da1bd8ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:34.200588 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc953df-f47c-4ad9-86af-0756da1bd8ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:34.202156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc953df-f47c-4ad9-86af-0756da1bd8ae admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:34.214382 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1100/2199] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:15:34.219905 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92789bfb-8cfe-4995-ab07-3495c6d8a00d admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:34.230420 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92789bfb-8cfe-4995-ab07-3495c6d8a00d admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:34.235722 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e8198-d53d-4d4a-84e8-1eeb902255af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:34.236395 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e8198-d53d-4d4a-84e8-1eeb902255af admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:34.236788 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e8198-d53d-4d4a-84e8-1eeb902255af admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:34.237066 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e8198-d53d-4d4a-84e8-1eeb902255af admin admin] PATH_INFO: `/v3/projects/8c64d1ec7ef949d4a4f48012de880664/users/c780d9a891064e11896d40f71015c946/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:34.254111 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e8198-d53d-4d4a-84e8-1eeb902255af admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c64d1ec7ef949d4a4f48012de880664, user_id=c780d9a891064e11896d40f71015c946, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:34.256907 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e8198-d53d-4d4a-84e8-1eeb902255af admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:34.283307 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1100/2200] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:15:34 2022] PUT /identity/v3/projects/8c64d1ec7ef949d4a4f48012de880664/users/c780d9a891064e11896d40f71015c946/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:15:34.287507 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0fdea0-79d0-4643-91ac-926674d174fb None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:34.287507 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0fdea0-79d0-4643-91ac-926674d174fb None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:34.287507 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0fdea0-79d0-4643-91ac-926674d174fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:34.325336 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-bd0fdea0-79d0-4643-91ac-926674d174fb None None] MFA Rules not processed for user `c780d9a891064e11896d40f71015c946`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:15:34.326703 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-bd0fdea0-79d0-4643-91ac-926674d174fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:34.402198 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1101/2201] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:15:34 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:15:34.414707 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e8198-d53d-4d4a-84e8-1eeb902255af admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:34.417325 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-d90e8198-d53d-4d4a-84e8-1eeb902255af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:34.438823 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:15:34.485855 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e8198-d53d-4d4a-84e8-1eeb902255af admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:34.498043 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6f9ca-7359-4e51-b2f8-3ee53245fce4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:34.501398 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6f9ca-7359-4e51-b2f8-3ee53245fce4 service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:34.502906 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6f9ca-7359-4e51-b2f8-3ee53245fce4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:34.503272 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6f9ca-7359-4e51-b2f8-3ee53245fce4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:34.523828 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d6f9ca-7359-4e51-b2f8-3ee53245fce4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:34.525288 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d6f9ca-7359-4e51-b2f8-3ee53245fce4 service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:34.544330 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1101/2202] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:15:34 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:15:44.399106 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=67,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:15:44.532165 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=20 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:15:54.191229 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0fdea0-79d0-4643-91ac-926674d174fb None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:54.250560 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0fdea0-79d0-4643-91ac-926674d174fb None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:54.255733 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d89a4a-d512-4b46-b613-843192f96bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:54.256368 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d89a4a-d512-4b46-b613-843192f96bc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:54.256649 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d89a4a-d512-4b46-b613-843192f96bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:54.256923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d89a4a-d512-4b46-b613-843192f96bc2 admin admin] PATH_INFO: `/v3/users/c780d9a891064e11896d40f71015c946` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:54.258043 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d89a4a-d512-4b46-b613-843192f96bc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c780d9a891064e11896d40f71015c946)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:54.259992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d89a4a-d512-4b46-b613-843192f96bc2 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:54.306918 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f9d89a4a-d512-4b46-b613-843192f96bc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c780d9a891064e11896d40f71015c946'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:54.315723 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f9d89a4a-d512-4b46-b613-843192f96bc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c780d9a891064e11896d40f71015c946'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:54.316323 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f9d89a4a-d512-4b46-b613-843192f96bc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c780d9a891064e11896d40f71015c946'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:54.316917 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f9d89a4a-d512-4b46-b613-843192f96bc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c780d9a891064e11896d40f71015c946'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:54.323808 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f9d89a4a-d512-4b46-b613-843192f96bc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c780d9a891064e11896d40f71015c946'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:54.341071 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1102/2203] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:15:54 2022] DELETE /identity/v3/users/c780d9a891064e11896d40f71015c946 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:15:54.345447 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01fe16-21b9-403c-a611-2123aa59fb45 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:54.345950 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01fe16-21b9-403c-a611-2123aa59fb45 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:54.346048 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01fe16-21b9-403c-a611-2123aa59fb45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:54.366222 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-6e01fe16-21b9-403c-a611-2123aa59fb45 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:15:54.367708 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-6e01fe16-21b9-403c-a611-2123aa59fb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:54.428006 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1102/2204] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:15:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:15:54.442574 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d89a4a-d512-4b46-b613-843192f96bc2 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:54.444037 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f9d89a4a-d512-4b46-b613-843192f96bc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:54.499697 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d89a4a-d512-4b46-b613-843192f96bc2 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:54.503815 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d00bc-e58f-4594-a551-9e772d93f724 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:54.504410 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d00bc-e58f-4594-a551-9e772d93f724 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:54.504681 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d00bc-e58f-4594-a551-9e772d93f724 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:54.504944 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d00bc-e58f-4594-a551-9e772d93f724 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:54.514098 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9d00bc-e58f-4594-a551-9e772d93f724 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:54.515548 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9d00bc-e58f-4594-a551-9e772d93f724 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:54.525415 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1103/2205] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:15:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:15:54.804163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e01fe16-21b9-403c-a611-2123aa59fb45 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:54.805757 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-6e01fe16-21b9-403c-a611-2123aa59fb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:54.830955 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e01fe16-21b9-403c-a611-2123aa59fb45 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:54.836979 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36c4df-1ed5-4506-8f28-35f06cf1fbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:54.837715 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36c4df-1ed5-4506-8f28-35f06cf1fbf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:54.838000 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36c4df-1ed5-4506-8f28-35f06cf1fbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:54.838262 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36c4df-1ed5-4506-8f28-35f06cf1fbf2 admin admin] PATH_INFO: `/v3/projects/8c64d1ec7ef949d4a4f48012de880664` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:54.839531 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb36c4df-1ed5-4506-8f28-35f06cf1fbf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c64d1ec7ef949d4a4f48012de880664)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:54.841432 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb36c4df-1ed5-4506-8f28-35f06cf1fbf2 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:54.890942 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-fb36c4df-1ed5-4506-8f28-35f06cf1fbf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c64d1ec7ef949d4a4f48012de880664', 'request_id': 'req-fb36c4df-1ed5-4506-8f28-35f06cf1fbf2'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:54.904079 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-fb36c4df-1ed5-4506-8f28-35f06cf1fbf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c64d1ec7ef949d4a4f48012de880664', 'request_id': 'req-fb36c4df-1ed5-4506-8f28-35f06cf1fbf2'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:54.913702 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-fb36c4df-1ed5-4506-8f28-35f06cf1fbf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c64d1ec7ef949d4a4f48012de880664', 'request_id': 'req-fb36c4df-1ed5-4506-8f28-35f06cf1fbf2'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:54.926193 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-fb36c4df-1ed5-4506-8f28-35f06cf1fbf2 admin admin] The token cache is being invalidate because project 8c64d1ec7ef949d4a4f48012de880664 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:15:54.926489 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-fb36c4df-1ed5-4506-8f28-35f06cf1fbf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:54.927201 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-fb36c4df-1ed5-4506-8f28-35f06cf1fbf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:15:54.929332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1103/2206] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:15:54 2022] DELETE /identity/v3/projects/8c64d1ec7ef949d4a4f48012de880664 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:15:54.934161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6bbd31-164b-4900-b573-c5fcbcba6ec8 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:54.934444 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6bbd31-164b-4900-b573-c5fcbcba6ec8 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:54.934714 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6bbd31-164b-4900-b573-c5fcbcba6ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:54.955346 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-df6bbd31-164b-4900-b573-c5fcbcba6ec8 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:15:54.956585 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-df6bbd31-164b-4900-b573-c5fcbcba6ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:54.994698 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:15:55.045648 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1104/2207] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:15:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:15:55.053671 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4639c-d8cd-45da-808c-5729ffaa2033 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:55.053953 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4639c-d8cd-45da-808c-5729ffaa2033 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:55.054229 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4639c-d8cd-45da-808c-5729ffaa2033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:55.075473 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-11a4639c-d8cd-45da-808c-5729ffaa2033 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:15:55.076722 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-11a4639c-d8cd-45da-808c-5729ffaa2033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:55.086874 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1104/2208] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:15:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:15:55.089573 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6bbd31-164b-4900-b573-c5fcbcba6ec8 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:55.099770 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6bbd31-164b-4900-b573-c5fcbcba6ec8 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:55.104680 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00ec87-a3e3-4495-9611-3dccae0821c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:55.105285 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00ec87-a3e3-4495-9611-3dccae0821c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:55.105565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00ec87-a3e3-4495-9611-3dccae0821c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:55.105841 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00ec87-a3e3-4495-9611-3dccae0821c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:55.106489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b00ec87-a3e3-4495-9611-3dccae0821c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:15:55.106791 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b00ec87-a3e3-4495-9611-3dccae0821c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:55.108404 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b00ec87-a3e3-4495-9611-3dccae0821c6 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:55.120955 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1105/2209] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:15:55.125318 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a4639c-d8cd-45da-808c-5729ffaa2033 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:55.134492 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a4639c-d8cd-45da-808c-5729ffaa2033 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:55.139212 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd5e32d-9cfb-4ed3-b0ae-81cbe4146871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:55.139806 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5e32d-9cfb-4ed3-b0ae-81cbe4146871 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:55.140082 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5e32d-9cfb-4ed3-b0ae-81cbe4146871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:55.140350 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5e32d-9cfb-4ed3-b0ae-81cbe4146871 admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:55.140987 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd5e32d-9cfb-4ed3-b0ae-81cbe4146871 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:55.142961 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd5e32d-9cfb-4ed3-b0ae-81cbe4146871 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:55.172699 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1105/2210] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:15:55 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:15:55.176962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00ec87-a3e3-4495-9611-3dccae0821c6 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:55.187124 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00ec87-a3e3-4495-9611-3dccae0821c6 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:55.244214 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55003e54-ee08-40e5-9bbf-f7af806e38e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:55.244898 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55003e54-ee08-40e5-9bbf-f7af806e38e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:55.245127 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55003e54-ee08-40e5-9bbf-f7af806e38e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:55.245348 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55003e54-ee08-40e5-9bbf-f7af806e38e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:55.245978 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55003e54-ee08-40e5-9bbf-f7af806e38e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:55.248024 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55003e54-ee08-40e5-9bbf-f7af806e38e7 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:55.270688 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1106/2211] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:15:55 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:15:55.274794 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd5e32d-9cfb-4ed3-b0ae-81cbe4146871 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:55.285685 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd5e32d-9cfb-4ed3-b0ae-81cbe4146871 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:55.290435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d3f1d-8611-4d3d-9b26-81a4ec0796fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:55.291143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d3f1d-8611-4d3d-9b26-81a4ec0796fc admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:55.291474 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d3f1d-8611-4d3d-9b26-81a4ec0796fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:55.291673 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d3f1d-8611-4d3d-9b26-81a4ec0796fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:55.292350 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268d3f1d-8611-4d3d-9b26-81a4ec0796fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:55.294347 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268d3f1d-8611-4d3d-9b26-81a4ec0796fc admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:55.307580 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1106/2212] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:15:55.308736 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55003e54-ee08-40e5-9bbf-f7af806e38e7 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:55.319388 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55003e54-ee08-40e5-9bbf-f7af806e38e7 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:55.324014 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95536107-4305-4a3e-a69c-a8e389837bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:55.324607 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95536107-4305-4a3e-a69c-a8e389837bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:55.324887 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95536107-4305-4a3e-a69c-a8e389837bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:55.325159 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95536107-4305-4a3e-a69c-a8e389837bc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:55.325801 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95536107-4305-4a3e-a69c-a8e389837bc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:55.327407 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95536107-4305-4a3e-a69c-a8e389837bc9 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:55.338641 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1107/2213] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:15:55.342890 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d3f1d-8611-4d3d-9b26-81a4ec0796fc admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:55.353633 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d3f1d-8611-4d3d-9b26-81a4ec0796fc admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:55.358275 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c9d6e-b7da-4585-bf1f-6d1d01521195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:55.358953 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c9d6e-b7da-4585-bf1f-6d1d01521195 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:55.359236 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c9d6e-b7da-4585-bf1f-6d1d01521195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:55.359510 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c9d6e-b7da-4585-bf1f-6d1d01521195 admin admin] PATH_INFO: `/v3/projects/40918a5eb3134a00999618cb65dedbbd/users/67c00c7b37ba44aa9ca0e4555d32ec90/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:55.373972 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c9d6e-b7da-4585-bf1f-6d1d01521195 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40918a5eb3134a00999618cb65dedbbd, user_id=67c00c7b37ba44aa9ca0e4555d32ec90, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:55.376839 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c9d6e-b7da-4585-bf1f-6d1d01521195 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:55.389474 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1107/2214] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:15:55 2022] PUT /identity/v3/projects/40918a5eb3134a00999618cb65dedbbd/users/67c00c7b37ba44aa9ca0e4555d32ec90/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:15:55.395433 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b002c-4c9b-4f96-8636-048c95d0143f None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:55.395719 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b002c-4c9b-4f96-8636-048c95d0143f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:55.395991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b002c-4c9b-4f96-8636-048c95d0143f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:55.424089 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-791b002c-4c9b-4f96-8636-048c95d0143f None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:15:55.425327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-791b002c-4c9b-4f96-8636-048c95d0143f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:55.496966 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1108/2215] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:15:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:15:55.507377 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c9d6e-b7da-4585-bf1f-6d1d01521195 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:55.509370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-bd0c9d6e-b7da-4585-bf1f-6d1d01521195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:55.536373 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:15:55.577536 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c9d6e-b7da-4585-bf1f-6d1d01521195 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:55.582257 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e98f030-9fa1-459f-b66f-8d0dda0c735f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:55.582956 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98f030-9fa1-459f-b66f-8d0dda0c735f service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:55.583237 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98f030-9fa1-459f-b66f-8d0dda0c735f service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:55.583510 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98f030-9fa1-459f-b66f-8d0dda0c735f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:55.593853 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e98f030-9fa1-459f-b66f-8d0dda0c735f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:55.595344 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e98f030-9fa1-459f-b66f-8d0dda0c735f service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:55.605708 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1108/2216] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:15:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:15:56.002166 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ae735-5277-4a98-a9ff-df8f14235ce6 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:56.002454 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ae735-5277-4a98-a9ff-df8f14235ce6 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:56.003423 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ae735-5277-4a98-a9ff-df8f14235ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:56.028035 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-815ae735-5277-4a98-a9ff-df8f14235ce6 None None] MFA Rules not processed for user `1405e758a3af4e8ba659f498cd28ace1`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:15:56.029247 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-815ae735-5277-4a98-a9ff-df8f14235ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:56.060067 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:15:56.095701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1109/2217] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Sep 12 14:15:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:15:56.107654 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e98f030-9fa1-459f-b66f-8d0dda0c735f service neutron] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:15:56.109099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-4e98f030-9fa1-459f-b66f-8d0dda0c735f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:15:56.180518 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e98f030-9fa1-459f-b66f-8d0dda0c735f service neutron] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:15:56.187113 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078ae29-b055-474f-b7fe-eb7b7e11bfbd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a87ea9c3c5409b97a35c4857ae3ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:15:56.187755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078ae29-b055-474f-b7fe-eb7b7e11bfbd service swift] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:15:56.188035 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078ae29-b055-474f-b7fe-eb7b7e11bfbd service swift] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:15:56.188568 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078ae29-b055-474f-b7fe-eb7b7e11bfbd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:15:56.197788 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5078ae29-b055-474f-b7fe-eb7b7e11bfbd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:15:56.199411 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5078ae29-b055-474f-b7fe-eb7b7e11bfbd service swift] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:15:56.207861 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1109/2218] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Sep 12 14:15:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 14:16:01.601544 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2b536-dadb-45f9-8a53-11e5bab876b0 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:16:01.602160 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2b536-dadb-45f9-8a53-11e5bab876b0 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:16:01.602160 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2b536-dadb-45f9-8a53-11e5bab876b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:16:01.649084 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-2ec2b536-dadb-45f9-8a53-11e5bab876b0 None None] MFA Rules not processed for user `67c00c7b37ba44aa9ca0e4555d32ec90`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:16:01.651039 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-2ec2b536-dadb-45f9-8a53-11e5bab876b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:16:01.803404 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1110/2219] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:16:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:16:01.808123 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078ae29-b055-474f-b7fe-eb7b7e11bfbd service swift] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:16:01.828723 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078ae29-b055-474f-b7fe-eb7b7e11bfbd service swift] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:16:01.840349 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c79645-e1e2-43d0-b867-b49451abe19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:16:01.844784 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c79645-e1e2-43d0-b867-b49451abe19c admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:16:01.845272 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c79645-e1e2-43d0-b867-b49451abe19c admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:16:01.852448 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c79645-e1e2-43d0-b867-b49451abe19c admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:16:01.853394 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c79645-e1e2-43d0-b867-b49451abe19c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:16:01.855511 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c79645-e1e2-43d0-b867-b49451abe19c admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:16:01.943279 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1110/2220] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:16:01 2022] POST /identity/v3/projects => generated 383 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:16:01.946992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2b536-dadb-45f9-8a53-11e5bab876b0 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:16:01.962333 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2b536-dadb-45f9-8a53-11e5bab876b0 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:16:02.131026 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f7982-2f1a-4937-8ba7-6cad3bc01b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:16:02.131683 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f7982-2f1a-4937-8ba7-6cad3bc01b7f admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:16:02.131937 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f7982-2f1a-4937-8ba7-6cad3bc01b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:16:02.132258 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f7982-2f1a-4937-8ba7-6cad3bc01b7f admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:16:02.132910 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f7982-2f1a-4937-8ba7-6cad3bc01b7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:16:02.136882 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f7982-2f1a-4937-8ba7-6cad3bc01b7f admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:16:02.175607 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1111/2221] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:16:01 2022] POST /identity/v3/users => generated 298 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:16:02.179348 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c79645-e1e2-43d0-b867-b49451abe19c admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:16:02.197320 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c79645-e1e2-43d0-b867-b49451abe19c admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:16:02.231856 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7afa484-0680-4217-aba3-df2c59fe8348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:16:02.232822 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afa484-0680-4217-aba3-df2c59fe8348 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:16:02.234731 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afa484-0680-4217-aba3-df2c59fe8348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:16:02.235823 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afa484-0680-4217-aba3-df2c59fe8348 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:16:02.238477 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7afa484-0680-4217-aba3-df2c59fe8348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:16:02.241008 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7afa484-0680-4217-aba3-df2c59fe8348 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:16:02.265994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1111/2222] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:16:02.270009 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f7982-2f1a-4937-8ba7-6cad3bc01b7f admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:16:02.290592 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f7982-2f1a-4937-8ba7-6cad3bc01b7f admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:16:02.303254 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b5ebd-1b14-459d-985c-0b98707687e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:16:02.303941 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b5ebd-1b14-459d-985c-0b98707687e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:16:02.304223 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b5ebd-1b14-459d-985c-0b98707687e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:16:02.304600 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b5ebd-1b14-459d-985c-0b98707687e9 admin admin] PATH_INFO: `/v3/projects/8f268344ee214486819057a236e65377/users/db6bb10286024e5f9cc3c880469e2b16/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:16:02.341639 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99b5ebd-1b14-459d-985c-0b98707687e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f268344ee214486819057a236e65377, user_id=db6bb10286024e5f9cc3c880469e2b16, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:16:02.341639 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99b5ebd-1b14-459d-985c-0b98707687e9 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:16:02.376149 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1112/2223] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:16:02 2022] PUT /identity/v3/projects/8f268344ee214486819057a236e65377/users/db6bb10286024e5f9cc3c880469e2b16/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:16:02.379828 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7afa484-0680-4217-aba3-df2c59fe8348 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:16:02.391941 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7afa484-0680-4217-aba3-df2c59fe8348 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:16:02.582691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebc4cb-62e3-4e37-8f58-246020e989ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:16:02.583367 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebc4cb-62e3-4e37-8f58-246020e989ee admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:16:02.583646 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebc4cb-62e3-4e37-8f58-246020e989ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:16:02.583923 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebc4cb-62e3-4e37-8f58-246020e989ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:16:02.584865 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ebc4cb-62e3-4e37-8f58-246020e989ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:16:02.598214 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ebc4cb-62e3-4e37-8f58-246020e989ee admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:16:02.629211 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1112/2224] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:16:02.630113 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b5ebd-1b14-459d-985c-0b98707687e9 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:16:02.644868 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b5ebd-1b14-459d-985c-0b98707687e9 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:16:02.658183 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6ded23-5a91-4249-81b5-7f7c940ebda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:16:02.659513 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ded23-5a91-4249-81b5-7f7c940ebda2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:16:02.659807 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ded23-5a91-4249-81b5-7f7c940ebda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:16:02.660526 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ded23-5a91-4249-81b5-7f7c940ebda2 admin admin] PATH_INFO: `/v3/projects/8f268344ee214486819057a236e65377/users/db6bb10286024e5f9cc3c880469e2b16/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:16:02.668406 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6ded23-5a91-4249-81b5-7f7c940ebda2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f268344ee214486819057a236e65377, user_id=db6bb10286024e5f9cc3c880469e2b16, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:16:02.671078 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6ded23-5a91-4249-81b5-7f7c940ebda2 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:16:02.709287 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1113/2225] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:16:02 2022] PUT /identity/v3/projects/8f268344ee214486819057a236e65377/users/db6bb10286024e5f9cc3c880469e2b16/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:16:09.097820 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aec0fe-3c5e-45dc-a2fb-4cd13efd8a76 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:16:09.098712 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aec0fe-3c5e-45dc-a2fb-4cd13efd8a76 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:16:09.099165 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aec0fe-3c5e-45dc-a2fb-4cd13efd8a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:16:09.182264 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-c8aec0fe-3c5e-45dc-a2fb-4cd13efd8a76 None None] MFA Rules not processed for user `db6bb10286024e5f9cc3c880469e2b16`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:16:09.184009 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-c8aec0fe-3c5e-45dc-a2fb-4cd13efd8a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:16:09.196435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:16:09.374651 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1113/2226] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:16:09 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:16:09.388453 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6ded23-5a91-4249-81b5-7f7c940ebda2 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:16:09.390618 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-bc6ded23-5a91-4249-81b5-7f7c940ebda2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:16:09.424248 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:16:09.489173 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6ded23-5a91-4249-81b5-7f7c940ebda2 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:16:09.497254 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62fda09-4b5e-4816-9c74-3a775f10699f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2092a9639067461c9a837a891f56889b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:16:09.497883 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62fda09-4b5e-4816-9c74-3a775f10699f service glance] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:16:09.498168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62fda09-4b5e-4816-9c74-3a775f10699f service glance] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:16:09.498440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62fda09-4b5e-4816-9c74-3a775f10699f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:16:09.610710 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62fda09-4b5e-4816-9c74-3a775f10699f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:16:09.613053 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62fda09-4b5e-4816-9c74-3a775f10699f service glance] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:16:09.624871 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1114/2227] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:16:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:16:09.815685 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5686f9-e86f-45e5-841d-0dbc95fbe377 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:16:09.816079 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5686f9-e86f-45e5-841d-0dbc95fbe377 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:16:09.816353 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5686f9-e86f-45e5-841d-0dbc95fbe377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:16:09.860875 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-5c5686f9-e86f-45e5-841d-0dbc95fbe377 None None] MFA Rules not processed for user `1405e758a3af4e8ba659f498cd28ace1`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:16:09.862050 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-5c5686f9-e86f-45e5-841d-0dbc95fbe377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:16:09.949599 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1114/2228] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Sep 12 14:16:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:16:09.969556 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62fda09-4b5e-4816-9c74-3a775f10699f service glance] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:16:10.063648 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62fda09-4b5e-4816-9c74-3a775f10699f service glance] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:16:10.069778 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a41533-6283-4894-98e2-343128f78542 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a87ea9c3c5409b97a35c4857ae3ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:16:10.070400 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a41533-6283-4894-98e2-343128f78542 service swift] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:16:10.070675 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a41533-6283-4894-98e2-343128f78542 service swift] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:16:10.070976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a41533-6283-4894-98e2-343128f78542 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:16:10.087711 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a41533-6283-4894-98e2-343128f78542 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:16:10.089218 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a41533-6283-4894-98e2-343128f78542 service swift] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:16:10.105873 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1115/2229] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Sep 12 14:16:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 12 14:16:19.194933 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5686f9-e86f-45e5-841d-0dbc95fbe377 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:16:19.196650 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-5c5686f9-e86f-45e5-841d-0dbc95fbe377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:16:19.257666 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a41533-6283-4894-98e2-343128f78542 service swift] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:16:19.260048 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-a1a41533-6283-4894-98e2-343128f78542 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:16:19.275684 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5686f9-e86f-45e5-841d-0dbc95fbe377 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:16:19.277789 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a41533-6283-4894-98e2-343128f78542 service swift] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:16:19.281449 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae9cacc-315b-48f1-9f18-c0ecfb77e2f3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce508bb9aa164a7fb1831a9b5ff5b932', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:16:19.282167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9cacc-315b-48f1-9f18-c0ecfb77e2f3 service placement] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:16:19.282451 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9cacc-315b-48f1-9f18-c0ecfb77e2f3 service placement] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:16:19.282732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9cacc-315b-48f1-9f18-c0ecfb77e2f3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:16:19.283447 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0473bd-6780-4227-95d3-500edb341c4e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce508bb9aa164a7fb1831a9b5ff5b932', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:16:19.284174 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0473bd-6780-4227-95d3-500edb341c4e service placement] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:16:19.284453 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0473bd-6780-4227-95d3-500edb341c4e service placement] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:16:19.284764 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0473bd-6780-4227-95d3-500edb341c4e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:16:19.285524 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-aae9cacc-315b-48f1-9f18-c0ecfb77e2f3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:16:19.290274 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-3e0473bd-6780-4227-95d3-500edb341c4e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:16:19.307048 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae9cacc-315b-48f1-9f18-c0ecfb77e2f3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:16:19.307048 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae9cacc-315b-48f1-9f18-c0ecfb77e2f3 service placement] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:16:19.316574 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0473bd-6780-4227-95d3-500edb341c4e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:16:19.318178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0473bd-6780-4227-95d3-500edb341c4e service placement] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:16:19.318948 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1115/2230] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:16:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:16:19.341643 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1116/2231] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:16:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:16:20.213380 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db4f06-9e36-417f-a5ea-6d0d309f4b01 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:16:20.214215 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db4f06-9e36-417f-a5ea-6d0d309f4b01 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:16:20.214754 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db4f06-9e36-417f-a5ea-6d0d309f4b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:16:20.236590 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-16db4f06-9e36-417f-a5ea-6d0d309f4b01 None None] MFA Rules not processed for user `67c00c7b37ba44aa9ca0e4555d32ec90`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:16:20.237986 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-16db4f06-9e36-417f-a5ea-6d0d309f4b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:16:20.245670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1116/2232] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:16:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:16:21.583994 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab063b-f7f9-40da-8d51-86d86d39ffa3 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:16:21.584393 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab063b-f7f9-40da-8d51-86d86d39ffa3 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:16:21.584581 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab063b-f7f9-40da-8d51-86d86d39ffa3 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:16:21.585934 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1117/2233] 10.222.0.42 () {54 vars in 921 bytes} [Mon Sep 12 14:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:16:21.592466 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8025099-ed57-470a-9d4d-53d97c3db5a9 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:16:21.592996 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8025099-ed57-470a-9d4d-53d97c3db5a9 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:16:21.593259 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8025099-ed57-470a-9d4d-53d97c3db5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:16:21.624760 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-d8025099-ed57-470a-9d4d-53d97c3db5a9 None None] MFA Rules not processed for user `854ab2dcec5a48b18f409b33cb61b5da`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:16:21.626275 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-d8025099-ed57-470a-9d4d-53d97c3db5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:16:21.660118 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:16:21.711779 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1117/2234] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Sep 12 14:16:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:16:21.725562 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab063b-f7f9-40da-8d51-86d86d39ffa3 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:16:21.728733 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-9fab063b-f7f9-40da-8d51-86d86d39ffa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:16:21.744509 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab063b-f7f9-40da-8d51-86d86d39ffa3 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:16:21.750035 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebb799c-4b86-4457-9a74-00db5af234b4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2092a9639067461c9a837a891f56889b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:16:21.750653 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb799c-4b86-4457-9a74-00db5af234b4 service glance] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:16:21.751475 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb799c-4b86-4457-9a74-00db5af234b4 service glance] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:16:21.752474 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb799c-4b86-4457-9a74-00db5af234b4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:16:21.765762 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebb799c-4b86-4457-9a74-00db5af234b4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:16:21.768005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebb799c-4b86-4457-9a74-00db5af234b4 service glance] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:16:21.779314 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1118/2235] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:16:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:16:21.871876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d877b-0c53-4829-a119-5f5a1ac27ed0 None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:16:21.873139 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d877b-0c53-4829-a119-5f5a1ac27ed0 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:16:21.873139 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d877b-0c53-4829-a119-5f5a1ac27ed0 None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:16:21.875095 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1118/2236] 10.222.0.42 () {54 vars in 921 bytes} [Mon Sep 12 14:16:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:16:21.880891 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c68a9e-6b49-41f0-a718-589fbe5b32ea None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:16:21.881180 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c68a9e-6b49-41f0-a718-589fbe5b32ea None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:16:21.881454 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c68a9e-6b49-41f0-a718-589fbe5b32ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:16:21.902442 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-04c68a9e-6b49-41f0-a718-589fbe5b32ea None None] MFA Rules not processed for user `865b1647468c4efe8efe4febe2203a58`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:16:21.904024 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-04c68a9e-6b49-41f0-a718-589fbe5b32ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:16:21.936709 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:16:21.968640 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1119/2237] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Sep 12 14:16:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:16:21.984153 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4d877b-0c53-4829-a119-5f5a1ac27ed0 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:16:21.996844 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4d877b-0c53-4829-a119-5f5a1ac27ed0 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:16:22.002666 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8745f-8373-412b-b74d-6c6f5fe8e038 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:16:22.003516 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8745f-8373-412b-b74d-6c6f5fe8e038 service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:16:22.004198 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8745f-8373-412b-b74d-6c6f5fe8e038 service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:16:22.004922 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8745f-8373-412b-b74d-6c6f5fe8e038 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:16:22.013446 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a8745f-8373-412b-b74d-6c6f5fe8e038 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:16:22.014888 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a8745f-8373-412b-b74d-6c6f5fe8e038 service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:16:22.025384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1119/2238] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:16:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:16:22.129217 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48158c52-79f1-4a89-8adc-9a81f0f3db9a None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:16:22.129441 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48158c52-79f1-4a89-8adc-9a81f0f3db9a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:16:22.129818 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48158c52-79f1-4a89-8adc-9a81f0f3db9a None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:16:22.131336 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1120/2239] 10.222.0.42 () {54 vars in 921 bytes} [Mon Sep 12 14:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:16:22.136645 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d42fa0-5b24-484f-bb76-e6c6df6056c8 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:16:22.136924 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d42fa0-5b24-484f-bb76-e6c6df6056c8 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:16:22.137191 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d42fa0-5b24-484f-bb76-e6c6df6056c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:16:22.157827 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-65d42fa0-5b24-484f-bb76-e6c6df6056c8 None None] MFA Rules not processed for user `854ab2dcec5a48b18f409b33cb61b5da`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:16:22.159014 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-65d42fa0-5b24-484f-bb76-e6c6df6056c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:16:22.168275 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1120/2240] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Sep 12 14:16:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:16:22.179672 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48158c52-79f1-4a89-8adc-9a81f0f3db9a None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:16:22.181417 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-48158c52-79f1-4a89-8adc-9a81f0f3db9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:16:22.195210 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48158c52-79f1-4a89-8adc-9a81f0f3db9a None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:16:22.198930 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a589a91-67f8-4162-82be-a64b49f9dfae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:16:22.199531 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a589a91-67f8-4162-82be-a64b49f9dfae service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:16:22.199813 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a589a91-67f8-4162-82be-a64b49f9dfae service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:16:22.200086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a589a91-67f8-4162-82be-a64b49f9dfae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:16:22.210324 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a589a91-67f8-4162-82be-a64b49f9dfae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:16:22.211774 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a589a91-67f8-4162-82be-a64b49f9dfae service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:16:22.223011 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1121/2241] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:16:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:16:24.786640 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deac78f-5ffa-4329-adec-f8b65cfd73bc None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:16:24.787200 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deac78f-5ffa-4329-adec-f8b65cfd73bc None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:16:24.787921 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deac78f-5ffa-4329-adec-f8b65cfd73bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:16:24.809968 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-0deac78f-5ffa-4329-adec-f8b65cfd73bc None None] MFA Rules not processed for user `1405e758a3af4e8ba659f498cd28ace1`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:16:24.811289 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-0deac78f-5ffa-4329-adec-f8b65cfd73bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:16:24.822545 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1121/2242] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Sep 12 14:16:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:16:24.836450 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a589a91-67f8-4162-82be-a64b49f9dfae service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:16:24.856378 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a589a91-67f8-4162-82be-a64b49f9dfae service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:16:24.864441 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a5259-27bf-4ece-ac4f-55f7803115c1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a87ea9c3c5409b97a35c4857ae3ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:16:24.865545 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a5259-27bf-4ece-ac4f-55f7803115c1 service swift] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:16:24.866509 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a5259-27bf-4ece-ac4f-55f7803115c1 service swift] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:16:24.867587 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a5259-27bf-4ece-ac4f-55f7803115c1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:16:24.878931 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a5259-27bf-4ece-ac4f-55f7803115c1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:16:24.880521 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a5259-27bf-4ece-ac4f-55f7803115c1 service swift] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:16:24.890725 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1122/2243] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Sep 12 14:16:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 14:16:34.803268 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=30 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:16:34.887365 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=20 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:16:40.013953 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c43b6c-217a-4ead-9b05-6dd916b3b0e8 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:16:40.014414 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c43b6c-217a-4ead-9b05-6dd916b3b0e8 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:16:40.014533 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c43b6c-217a-4ead-9b05-6dd916b3b0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:16:40.035978 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-76c43b6c-217a-4ead-9b05-6dd916b3b0e8 None None] MFA Rules not processed for user `1405e758a3af4e8ba659f498cd28ace1`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:16:40.037242 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-76c43b6c-217a-4ead-9b05-6dd916b3b0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:16:40.044371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1122/2244] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Sep 12 14:16:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:16:40.056738 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a5259-27bf-4ece-ac4f-55f7803115c1 service swift] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:16:40.067802 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a5259-27bf-4ece-ac4f-55f7803115c1 service swift] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:16:40.072916 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf41634-ffe2-406f-a625-5f7b44dc58f0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a87ea9c3c5409b97a35c4857ae3ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:16:40.073495 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf41634-ffe2-406f-a625-5f7b44dc58f0 service swift] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:16:40.073769 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf41634-ffe2-406f-a625-5f7b44dc58f0 service swift] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:16:40.074037 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf41634-ffe2-406f-a625-5f7b44dc58f0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:16:40.083497 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf41634-ffe2-406f-a625-5f7b44dc58f0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:16:40.086246 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf41634-ffe2-406f-a625-5f7b44dc58f0 service swift] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:16:40.094435 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1123/2245] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Sep 12 14:16:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 14:16:45.120716 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975a28f-aaa6-4a49-8e5f-15eb8e9feb57 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:16:45.121284 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975a28f-aaa6-4a49-8e5f-15eb8e9feb57 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:16:45.121284 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975a28f-aaa6-4a49-8e5f-15eb8e9feb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:16:45.140617 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-c975a28f-aaa6-4a49-8e5f-15eb8e9feb57 None None] MFA Rules not processed for user `02fd338d362a461a846a454c487bf669`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:16:45.142554 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-c975a28f-aaa6-4a49-8e5f-15eb8e9feb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:16:45.201464 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1123/2246] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:16:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:16:50.091225 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=6 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:16:55.194886 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=28 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:17:03.546952 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a7632-64cf-43b7-9bd8-ae54aaf0e917 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:17:03.547473 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a7632-64cf-43b7-9bd8-ae54aaf0e917 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:17:03.547473 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a7632-64cf-43b7-9bd8-ae54aaf0e917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:17:03.567984 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-e14a7632-64cf-43b7-9bd8-ae54aaf0e917 None None] MFA Rules not processed for user `1405e758a3af4e8ba659f498cd28ace1`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:17:03.569190 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e14a7632-64cf-43b7-9bd8-ae54aaf0e917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:17:03.577713 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1124/2247] 10.222.0.42 () {58 vars in 967 bytes} [Mon Sep 12 14:17:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:17:03.592144 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c975a28f-aaa6-4a49-8e5f-15eb8e9feb57 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:17:03.614984 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c975a28f-aaa6-4a49-8e5f-15eb8e9feb57 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:17:03.621784 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf39cbcf-1a8c-4066-bed1-fc8ad1586f1f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a87ea9c3c5409b97a35c4857ae3ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:17:03.622398 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39cbcf-1a8c-4066-bed1-fc8ad1586f1f service swift] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:17:03.622678 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39cbcf-1a8c-4066-bed1-fc8ad1586f1f service swift] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:17:03.622976 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39cbcf-1a8c-4066-bed1-fc8ad1586f1f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:17:03.636052 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf39cbcf-1a8c-4066-bed1-fc8ad1586f1f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:17:03.638002 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf39cbcf-1a8c-4066-bed1-fc8ad1586f1f service swift] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:17:03.657204 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1124/2248] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Sep 12 14:17:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 14:17:07.198452 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a7632-64cf-43b7-9bd8-ae54aaf0e917 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:17:07.207572 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a7632-64cf-43b7-9bd8-ae54aaf0e917 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:17:07.211070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02144d8d-7b78-4d85-9a4f-a6601c20ca23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:17:07.211661 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02144d8d-7b78-4d85-9a4f-a6601c20ca23 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:17:07.211935 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02144d8d-7b78-4d85-9a4f-a6601c20ca23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:17:07.212202 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02144d8d-7b78-4d85-9a4f-a6601c20ca23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:17:07.221333 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02144d8d-7b78-4d85-9a4f-a6601c20ca23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:17:07.222724 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02144d8d-7b78-4d85-9a4f-a6601c20ca23 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:17:07.231818 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1125/2249] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:17:07 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:17:13.650269 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=6 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:17:14.395760 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb808be1-8a7a-4494-9602-28fa01e7adfa None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:17:14.396043 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb808be1-8a7a-4494-9602-28fa01e7adfa None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:17:14.396335 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb808be1-8a7a-4494-9602-28fa01e7adfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:17:14.414771 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-bb808be1-8a7a-4494-9602-28fa01e7adfa None None] MFA Rules not processed for user `fff8e93029f447bb8294d9b516eb407f`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:17:14.415988 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-bb808be1-8a7a-4494-9602-28fa01e7adfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:17:14.472773 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1125/2250] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:17:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:17:17.227728 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=9 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:17:24.466276 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=25 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:17:25.200087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e899a-a52f-415c-952d-02698399a576 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:17:25.201508 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e899a-a52f-415c-952d-02698399a576 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:17:25.201945 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e899a-a52f-415c-952d-02698399a576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:17:25.231443 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-111e899a-a52f-415c-952d-02698399a576 None None] MFA Rules not processed for user `5f2ff5e497b64e49b7ac25a941a06048`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:17:25.232826 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-111e899a-a52f-415c-952d-02698399a576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:17:25.311321 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1126/2251] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:17:25 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:17:35.303780 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=25 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:17:59.754514 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb808be1-8a7a-4494-9602-28fa01e7adfa None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:17:59.756074 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-bb808be1-8a7a-4494-9602-28fa01e7adfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:17:59.821056 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb808be1-8a7a-4494-9602-28fa01e7adfa None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:17:59.825513 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414a22b-e04f-4220-ade5-6ad52fe497fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:17:59.826153 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414a22b-e04f-4220-ade5-6ad52fe497fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:17:59.826413 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414a22b-e04f-4220-ade5-6ad52fe497fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:17:59.826702 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414a22b-e04f-4220-ade5-6ad52fe497fc admin admin] PATH_INFO: `/v3/users/02fd338d362a461a846a454c487bf669` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:17:59.827820 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414a22b-e04f-4220-ade5-6ad52fe497fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=02fd338d362a461a846a454c487bf669)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:17:59.829773 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414a22b-e04f-4220-ade5-6ad52fe497fc admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:17:59.874655 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-c414a22b-e04f-4220-ade5-6ad52fe497fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02fd338d362a461a846a454c487bf669'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:17:59.881123 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-c414a22b-e04f-4220-ade5-6ad52fe497fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02fd338d362a461a846a454c487bf669'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:17:59.881820 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-c414a22b-e04f-4220-ade5-6ad52fe497fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02fd338d362a461a846a454c487bf669'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:17:59.882462 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-c414a22b-e04f-4220-ade5-6ad52fe497fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02fd338d362a461a846a454c487bf669'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:17:59.889282 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-c414a22b-e04f-4220-ade5-6ad52fe497fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02fd338d362a461a846a454c487bf669'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:17:59.905916 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1126/2252] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:17:59 2022] DELETE /identity/v3/users/02fd338d362a461a846a454c487bf669 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:18:00.078170 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111e899a-a52f-415c-952d-02698399a576 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:00.093044 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111e899a-a52f-415c-952d-02698399a576 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:00.155010 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a1b6a-8b72-4385-8c61-871e73f6ab24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:00.155646 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a1b6a-8b72-4385-8c61-871e73f6ab24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:00.155925 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a1b6a-8b72-4385-8c61-871e73f6ab24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:00.156198 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a1b6a-8b72-4385-8c61-871e73f6ab24 admin admin] PATH_INFO: `/v3/projects/34866fb3f98042d38c922bf3552c4e69` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:00.157872 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a1b6a-8b72-4385-8c61-871e73f6ab24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34866fb3f98042d38c922bf3552c4e69)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:00.159794 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a1b6a-8b72-4385-8c61-871e73f6ab24 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:00.211183 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-193a1b6a-8b72-4385-8c61-871e73f6ab24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34866fb3f98042d38c922bf3552c4e69', 'request_id': 'req-193a1b6a-8b72-4385-8c61-871e73f6ab24'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:00.222695 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-193a1b6a-8b72-4385-8c61-871e73f6ab24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34866fb3f98042d38c922bf3552c4e69', 'request_id': 'req-193a1b6a-8b72-4385-8c61-871e73f6ab24'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:00.230210 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-193a1b6a-8b72-4385-8c61-871e73f6ab24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34866fb3f98042d38c922bf3552c4e69', 'request_id': 'req-193a1b6a-8b72-4385-8c61-871e73f6ab24'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:00.242858 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-193a1b6a-8b72-4385-8c61-871e73f6ab24 admin admin] The token cache is being invalidate because project 34866fb3f98042d38c922bf3552c4e69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:18:00.243249 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-193a1b6a-8b72-4385-8c61-871e73f6ab24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:00.243938 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-193a1b6a-8b72-4385-8c61-871e73f6ab24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:00.245470 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1127/2253] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:18:00 2022] DELETE /identity/v3/projects/34866fb3f98042d38c922bf3552c4e69 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:18:02.429101 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66dcdc-ad1c-43e7-bf99-b8340ff6d6ba None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:02.429603 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66dcdc-ad1c-43e7-bf99-b8340ff6d6ba None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:02.429716 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66dcdc-ad1c-43e7-bf99-b8340ff6d6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:02.449640 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-6f66dcdc-ad1c-43e7-bf99-b8340ff6d6ba None None] MFA Rules not processed for user `0cafcc70e0e74a5fa7c5ce530e0e848f`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:18:02.450944 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-6f66dcdc-ad1c-43e7-bf99-b8340ff6d6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:02.504864 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1127/2254] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:18:02 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:18:10.028319 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a1b6a-8b72-4385-8c61-871e73f6ab24 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:10.029807 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-193a1b6a-8b72-4385-8c61-871e73f6ab24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:10.085943 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a1b6a-8b72-4385-8c61-871e73f6ab24 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:10.089445 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ceda8-59c0-46c9-9319-2e8ff327239f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:10.090034 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ceda8-59c0-46c9-9319-2e8ff327239f service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:10.090309 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ceda8-59c0-46c9-9319-2e8ff327239f service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:10.090576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ceda8-59c0-46c9-9319-2e8ff327239f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:10.092273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e21ceda8-59c0-46c9-9319-2e8ff327239f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:10.137160 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:18:10.164918 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ceda8-59c0-46c9-9319-2e8ff327239f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:10.166327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ceda8-59c0-46c9-9319-2e8ff327239f service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:10.177209 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1128/2255] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:18:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:18:11.863885 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f66dcdc-ad1c-43e7-bf99-b8340ff6d6ba None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:11.865268 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-6f66dcdc-ad1c-43e7-bf99-b8340ff6d6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:11.893094 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:18:11.922195 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f66dcdc-ad1c-43e7-bf99-b8340ff6d6ba None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:11.925726 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f40e07-5f8c-4ae7-a3e8-d438da4bc6d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:11.926316 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f40e07-5f8c-4ae7-a3e8-d438da4bc6d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:11.926593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f40e07-5f8c-4ae7-a3e8-d438da4bc6d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:11.926886 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f40e07-5f8c-4ae7-a3e8-d438da4bc6d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:11.928592 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-74f40e07-5f8c-4ae7-a3e8-d438da4bc6d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:11.942221 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f40e07-5f8c-4ae7-a3e8-d438da4bc6d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:11.943630 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f40e07-5f8c-4ae7-a3e8-d438da4bc6d8 service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:11.954709 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1128/2256] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:18:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:18:20.172506 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=13 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:18:21.256899 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ceda8-59c0-46c9-9319-2e8ff327239f service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:21.258607 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e21ceda8-59c0-46c9-9319-2e8ff327239f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:21.273287 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ceda8-59c0-46c9-9319-2e8ff327239f service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:21.285035 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae2a8c-7f73-4345-a2ea-7472aba09c18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:21.285684 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae2a8c-7f73-4345-a2ea-7472aba09c18 service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:21.285971 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae2a8c-7f73-4345-a2ea-7472aba09c18 service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:21.286641 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae2a8c-7f73-4345-a2ea-7472aba09c18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:21.288759 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-77ae2a8c-7f73-4345-a2ea-7472aba09c18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:21.423855 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae2a8c-7f73-4345-a2ea-7472aba09c18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:21.427831 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae2a8c-7f73-4345-a2ea-7472aba09c18 service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:21.452319 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1129/2257] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:18:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:18:21.949803 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=15 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:18:25.668873 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ead00-7597-44b4-9a2e-98bdc03c38c3 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:25.669465 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ead00-7597-44b4-9a2e-98bdc03c38c3 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:25.669465 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ead00-7597-44b4-9a2e-98bdc03c38c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:25.688161 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-163ead00-7597-44b4-9a2e-98bdc03c38c3 None None] MFA Rules not processed for user `e7cb679ed52e42ca83a2084f1ad4bd92`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:18:25.689333 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-163ead00-7597-44b4-9a2e-98bdc03c38c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:25.696121 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1129/2258] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:18:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:18:28.522739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae2a8c-7f73-4345-a2ea-7472aba09c18 service nova] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:28.524260 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-77ae2a8c-7f73-4345-a2ea-7472aba09c18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:28.539860 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae2a8c-7f73-4345-a2ea-7472aba09c18 service nova] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:28.545621 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3e1960-a4a9-40b6-ba82-0225e94bc1aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:28.546243 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e1960-a4a9-40b6-ba82-0225e94bc1aa service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:28.546525 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e1960-a4a9-40b6-ba82-0225e94bc1aa service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:28.546797 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e1960-a4a9-40b6-ba82-0225e94bc1aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:28.549543 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ead00-7597-44b4-9a2e-98bdc03c38c3 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:28.550008 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-dd3e1960-a4a9-40b6-ba82-0225e94bc1aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:28.552816 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-163ead00-7597-44b4-9a2e-98bdc03c38c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:28.569971 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ead00-7597-44b4-9a2e-98bdc03c38c3 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:28.573680 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e25ba5-4f19-41b8-a12a-a2985778266c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:28.574515 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e25ba5-4f19-41b8-a12a-a2985778266c service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:28.574992 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e25ba5-4f19-41b8-a12a-a2985778266c service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:28.575438 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e25ba5-4f19-41b8-a12a-a2985778266c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:28.577550 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-11e25ba5-4f19-41b8-a12a-a2985778266c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:28.642980 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3e1960-a4a9-40b6-ba82-0225e94bc1aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:28.645315 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3e1960-a4a9-40b6-ba82-0225e94bc1aa service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:28.652137 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e25ba5-4f19-41b8-a12a-a2985778266c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:28.653561 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e25ba5-4f19-41b8-a12a-a2985778266c service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:28.655789 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1130/2259] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:18:28 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:18:28.664461 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1130/2260] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:18:28 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:18:29.991491 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3e1960-a4a9-40b6-ba82-0225e94bc1aa service nova] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:29.993325 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-dd3e1960-a4a9-40b6-ba82-0225e94bc1aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:30.024288 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3e1960-a4a9-40b6-ba82-0225e94bc1aa service nova] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:30.028571 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee2631-b3b7-4c8d-b6f3-b9a5d9e7a314 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:30.029190 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee2631-b3b7-4c8d-b6f3-b9a5d9e7a314 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:30.029475 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee2631-b3b7-4c8d-b6f3-b9a5d9e7a314 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:30.029740 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee2631-b3b7-4c8d-b6f3-b9a5d9e7a314 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:30.032126 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1aee2631-b3b7-4c8d-b6f3-b9a5d9e7a314 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:30.127942 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aee2631-b3b7-4c8d-b6f3-b9a5d9e7a314 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:30.133421 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aee2631-b3b7-4c8d-b6f3-b9a5d9e7a314 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:30.159425 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1131/2261] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:18:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:18:38.659343 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=31 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:18:39.365536 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e25ba5-4f19-41b8-a12a-a2985778266c service neutron] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:39.377834 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e25ba5-4f19-41b8-a12a-a2985778266c service neutron] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:39.382149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda25e2-3637-4edf-bb20-cb02033f7eb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:39.382918 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda25e2-3637-4edf-bb20-cb02033f7eb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:39.383300 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda25e2-3637-4edf-bb20-cb02033f7eb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:39.383660 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda25e2-3637-4edf-bb20-cb02033f7eb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:39.385807 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-8bda25e2-3637-4edf-bb20-cb02033f7eb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:39.459467 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda25e2-3637-4edf-bb20-cb02033f7eb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:39.461159 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda25e2-3637-4edf-bb20-cb02033f7eb5 service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:39.471778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1131/2262] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:18:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:18:39.518084 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee2631-b3b7-4c8d-b6f3-b9a5d9e7a314 service cinder] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:39.528356 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee2631-b3b7-4c8d-b6f3-b9a5d9e7a314 service cinder] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:39.533798 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392338dd-016a-44d1-bf22-7c60ffc1fcc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:39.534424 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392338dd-016a-44d1-bf22-7c60ffc1fcc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:39.534695 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392338dd-016a-44d1-bf22-7c60ffc1fcc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:39.535004 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392338dd-016a-44d1-bf22-7c60ffc1fcc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:39.536739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-392338dd-016a-44d1-bf22-7c60ffc1fcc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:39.547223 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392338dd-016a-44d1-bf22-7c60ffc1fcc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:39.548664 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392338dd-016a-44d1-bf22-7c60ffc1fcc4 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:39.557683 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1132/2263] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:18:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:18:43.185230 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda25e2-3637-4edf-bb20-cb02033f7eb5 service neutron] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:43.186976 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-8bda25e2-3637-4edf-bb20-cb02033f7eb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:43.205698 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda25e2-3637-4edf-bb20-cb02033f7eb5 service neutron] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:43.210677 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f195dc17-0f95-4f64-8c54-bcb9bd347d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:43.211987 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195dc17-0f95-4f64-8c54-bcb9bd347d88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:43.212271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195dc17-0f95-4f64-8c54-bcb9bd347d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:43.212544 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195dc17-0f95-4f64-8c54-bcb9bd347d88 admin admin] PATH_INFO: `/v3/users/5f2ff5e497b64e49b7ac25a941a06048` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:43.213742 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f195dc17-0f95-4f64-8c54-bcb9bd347d88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f2ff5e497b64e49b7ac25a941a06048)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:43.218890 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f195dc17-0f95-4f64-8c54-bcb9bd347d88 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:43.278304 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-f195dc17-0f95-4f64-8c54-bcb9bd347d88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f2ff5e497b64e49b7ac25a941a06048'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:43.290968 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-f195dc17-0f95-4f64-8c54-bcb9bd347d88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f2ff5e497b64e49b7ac25a941a06048'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:43.292467 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-f195dc17-0f95-4f64-8c54-bcb9bd347d88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f2ff5e497b64e49b7ac25a941a06048'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:43.293409 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-f195dc17-0f95-4f64-8c54-bcb9bd347d88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f2ff5e497b64e49b7ac25a941a06048'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:43.301452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-f195dc17-0f95-4f64-8c54-bcb9bd347d88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f2ff5e497b64e49b7ac25a941a06048'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:43.335381 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1132/2264] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:18:43 2022] DELETE /identity/v3/users/5f2ff5e497b64e49b7ac25a941a06048 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:18:43.512382 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392338dd-016a-44d1-bf22-7c60ffc1fcc4 service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:43.525962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392338dd-016a-44d1-bf22-7c60ffc1fcc4 service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:43.546821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:18:43.597754 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f38ec-c82b-4ba0-a3a9-45f8ae018ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:43.598502 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f38ec-c82b-4ba0-a3a9-45f8ae018ca8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:43.598785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f38ec-c82b-4ba0-a3a9-45f8ae018ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:43.599086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f38ec-c82b-4ba0-a3a9-45f8ae018ca8 admin admin] PATH_INFO: `/v3/projects/366826b14e0e4a93819c18d8d716a254` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:43.600196 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f38ec-c82b-4ba0-a3a9-45f8ae018ca8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=366826b14e0e4a93819c18d8d716a254)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:43.602320 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f38ec-c82b-4ba0-a3a9-45f8ae018ca8 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:43.671466 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-8a9f38ec-c82b-4ba0-a3a9-45f8ae018ca8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '366826b14e0e4a93819c18d8d716a254', 'request_id': 'req-8a9f38ec-c82b-4ba0-a3a9-45f8ae018ca8'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:43.686786 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-8a9f38ec-c82b-4ba0-a3a9-45f8ae018ca8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '366826b14e0e4a93819c18d8d716a254', 'request_id': 'req-8a9f38ec-c82b-4ba0-a3a9-45f8ae018ca8'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:43.699660 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-8a9f38ec-c82b-4ba0-a3a9-45f8ae018ca8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '366826b14e0e4a93819c18d8d716a254', 'request_id': 'req-8a9f38ec-c82b-4ba0-a3a9-45f8ae018ca8'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:43.718568 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-8a9f38ec-c82b-4ba0-a3a9-45f8ae018ca8 admin admin] The token cache is being invalidate because project 366826b14e0e4a93819c18d8d716a254 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:18:43.719024 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-8a9f38ec-c82b-4ba0-a3a9-45f8ae018ca8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:43.719984 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-8a9f38ec-c82b-4ba0-a3a9-45f8ae018ca8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:43.722058 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1133/2265] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:18:43 2022] DELETE /identity/v3/projects/366826b14e0e4a93819c18d8d716a254 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:18:43.727992 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b861e2b-49aa-418d-800d-ed924491e21d None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:43.728460 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b861e2b-49aa-418d-800d-ed924491e21d None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:43.729822 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b861e2b-49aa-418d-800d-ed924491e21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:43.760511 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-6b861e2b-49aa-418d-800d-ed924491e21d None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:18:43.761737 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-6b861e2b-49aa-418d-800d-ed924491e21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:43.842585 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1133/2266] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:18:43.852587 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703b027-fbfe-49f7-bd14-53905724fa80 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:43.853605 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703b027-fbfe-49f7-bd14-53905724fa80 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:43.853998 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703b027-fbfe-49f7-bd14-53905724fa80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:43.880906 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-b703b027-fbfe-49f7-bd14-53905724fa80 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:18:43.882203 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b703b027-fbfe-49f7-bd14-53905724fa80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:43.894883 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1134/2267] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:18:43.898706 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b861e2b-49aa-418d-800d-ed924491e21d None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:43.912911 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b861e2b-49aa-418d-800d-ed924491e21d None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:43.918812 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd905ac8-37c1-4d0d-a5ac-25b6c9c3f3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:43.919760 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd905ac8-37c1-4d0d-a5ac-25b6c9c3f3c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:43.920038 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd905ac8-37c1-4d0d-a5ac-25b6c9c3f3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:43.920380 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd905ac8-37c1-4d0d-a5ac-25b6c9c3f3c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:43.921247 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd905ac8-37c1-4d0d-a5ac-25b6c9c3f3c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:18:43.921612 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd905ac8-37c1-4d0d-a5ac-25b6c9c3f3c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:43.923943 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd905ac8-37c1-4d0d-a5ac-25b6c9c3f3c6 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:43.925757 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:18:43.949611 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1134/2268] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:18:43.953621 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b703b027-fbfe-49f7-bd14-53905724fa80 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:43.964933 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b703b027-fbfe-49f7-bd14-53905724fa80 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:43.972598 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e4410-fc28-46cc-8a3f-cc9f370f2e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:43.974155 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e4410-fc28-46cc-8a3f-cc9f370f2e9e admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:43.974649 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e4410-fc28-46cc-8a3f-cc9f370f2e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:43.975172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e4410-fc28-46cc-8a3f-cc9f370f2e9e admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:43.976081 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2e4410-fc28-46cc-8a3f-cc9f370f2e9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:43.978273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2e4410-fc28-46cc-8a3f-cc9f370f2e9e admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:44.028505 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1135/2269] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:18:43 2022] POST /identity/v3/projects => generated 385 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:18:44.030123 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd905ac8-37c1-4d0d-a5ac-25b6c9c3f3c6 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:44.043622 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd905ac8-37c1-4d0d-a5ac-25b6c9c3f3c6 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:44.126733 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b397f9-9b02-4f4a-afaf-7d39d1b86614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:44.129346 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b397f9-9b02-4f4a-afaf-7d39d1b86614 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:44.129803 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b397f9-9b02-4f4a-afaf-7d39d1b86614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:44.130257 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b397f9-9b02-4f4a-afaf-7d39d1b86614 admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:44.131806 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b397f9-9b02-4f4a-afaf-7d39d1b86614 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:44.135417 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b397f9-9b02-4f4a-afaf-7d39d1b86614 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:44.165601 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1135/2270] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:18:44 2022] POST /identity/v3/users => generated 293 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:18:44.169594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e4410-fc28-46cc-8a3f-cc9f370f2e9e admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:44.182220 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e4410-fc28-46cc-8a3f-cc9f370f2e9e admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:44.189611 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1459f802-ece1-4274-b452-0c8be9a55cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:44.190462 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459f802-ece1-4274-b452-0c8be9a55cd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:44.190822 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459f802-ece1-4274-b452-0c8be9a55cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:44.191815 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459f802-ece1-4274-b452-0c8be9a55cd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:44.192556 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1459f802-ece1-4274-b452-0c8be9a55cd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:44.194256 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1459f802-ece1-4274-b452-0c8be9a55cd0 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:44.207074 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1136/2271] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:18:44.211338 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b397f9-9b02-4f4a-afaf-7d39d1b86614 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:44.226489 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b397f9-9b02-4f4a-afaf-7d39d1b86614 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:44.235578 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8debae-c2f1-415e-859a-d988e3d69856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:44.236579 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8debae-c2f1-415e-859a-d988e3d69856 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:44.237208 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8debae-c2f1-415e-859a-d988e3d69856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:44.237935 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8debae-c2f1-415e-859a-d988e3d69856 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:44.238892 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8debae-c2f1-415e-859a-d988e3d69856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:44.241303 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8debae-c2f1-415e-859a-d988e3d69856 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:44.263242 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1136/2272] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:18:44.267568 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1459f802-ece1-4274-b452-0c8be9a55cd0 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:44.283479 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1459f802-ece1-4274-b452-0c8be9a55cd0 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:44.290089 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda728a-9d66-461c-a2e6-836131c9e205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:44.291176 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda728a-9d66-461c-a2e6-836131c9e205 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:44.291460 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda728a-9d66-461c-a2e6-836131c9e205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:44.291734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda728a-9d66-461c-a2e6-836131c9e205 admin admin] PATH_INFO: `/v3/projects/7983920985ad438a8725654371ffbe4f/users/e6555c0ef9444e40a87b6f2351b408c1/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:44.314906 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda728a-9d66-461c-a2e6-836131c9e205 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7983920985ad438a8725654371ffbe4f, user_id=e6555c0ef9444e40a87b6f2351b408c1, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:44.321976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda728a-9d66-461c-a2e6-836131c9e205 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:44.341544 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1137/2273] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:18:44 2022] PUT /identity/v3/projects/7983920985ad438a8725654371ffbe4f/users/e6555c0ef9444e40a87b6f2351b408c1/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:18:44.353261 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80df78-6952-445e-b4ec-a8daab66562b None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:44.353955 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80df78-6952-445e-b4ec-a8daab66562b None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:44.353955 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80df78-6952-445e-b4ec-a8daab66562b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:44.398353 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-fd80df78-6952-445e-b4ec-a8daab66562b None None] MFA Rules not processed for user `e6555c0ef9444e40a87b6f2351b408c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:18:44.399626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-fd80df78-6952-445e-b4ec-a8daab66562b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:44.524888 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1137/2274] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:18:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:18:44.535843 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda728a-9d66-461c-a2e6-836131c9e205 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:44.537817 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-cdda728a-9d66-461c-a2e6-836131c9e205 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:44.623953 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:18:44.657748 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda728a-9d66-461c-a2e6-836131c9e205 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:44.667583 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1500ba-5c18-43c9-a301-1d6d646aad35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:44.668548 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1500ba-5c18-43c9-a301-1d6d646aad35 service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:44.668968 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1500ba-5c18-43c9-a301-1d6d646aad35 service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:44.669651 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1500ba-5c18-43c9-a301-1d6d646aad35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:44.681714 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1500ba-5c18-43c9-a301-1d6d646aad35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:44.684372 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1500ba-5c18-43c9-a301-1d6d646aad35 service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:44.698263 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1138/2275] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:18:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:18:45.202824 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9848fd1-a4f3-418f-833e-2c295c426fbd None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:45.203354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9848fd1-a4f3-418f-833e-2c295c426fbd None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:45.203621 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9848fd1-a4f3-418f-833e-2c295c426fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:45.223274 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-d9848fd1-a4f3-418f-833e-2c295c426fbd None None] MFA Rules not processed for user `fff8e93029f447bb8294d9b516eb407f`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:18:45.224445 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-d9848fd1-a4f3-418f-833e-2c295c426fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:45.278550 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1138/2276] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:18:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:18:45.593657 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1500ba-5c18-43c9-a301-1d6d646aad35 service nova] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:45.595178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-bf1500ba-5c18-43c9-a301-1d6d646aad35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:45.653847 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1500ba-5c18-43c9-a301-1d6d646aad35 service nova] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:45.657474 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ecdd2-bf54-47d9-8d4b-fcafb95ac1c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:45.658659 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ecdd2-bf54-47d9-8d4b-fcafb95ac1c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:45.658659 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ecdd2-bf54-47d9-8d4b-fcafb95ac1c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:45.658659 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ecdd2-bf54-47d9-8d4b-fcafb95ac1c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:45.660406 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-237ecdd2-bf54-47d9-8d4b-fcafb95ac1c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:45.669989 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ecdd2-bf54-47d9-8d4b-fcafb95ac1c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:45.671462 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ecdd2-bf54-47d9-8d4b-fcafb95ac1c6 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:45.679917 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1139/2277] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:18:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:18:46.167472 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9848fd1-a4f3-418f-833e-2c295c426fbd None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:46.171926 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-d9848fd1-a4f3-418f-833e-2c295c426fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:46.199215 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:18:46.299135 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9848fd1-a4f3-418f-833e-2c295c426fbd None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:46.306050 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c16ae-2d8d-474b-8bdf-03e2423a8134 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce508bb9aa164a7fb1831a9b5ff5b932', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:46.306674 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c16ae-2d8d-474b-8bdf-03e2423a8134 service placement] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:46.306984 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c16ae-2d8d-474b-8bdf-03e2423a8134 service placement] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:46.307254 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c16ae-2d8d-474b-8bdf-03e2423a8134 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:46.309291 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-ba2c16ae-2d8d-474b-8bdf-03e2423a8134 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:46.329394 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2c16ae-2d8d-474b-8bdf-03e2423a8134 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:46.331009 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2c16ae-2d8d-474b-8bdf-03e2423a8134 service placement] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:46.343410 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1139/2278] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:18:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:18:51.993607 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ecdd2-bf54-47d9-8d4b-fcafb95ac1c6 service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:52.017769 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ecdd2-bf54-47d9-8d4b-fcafb95ac1c6 service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:52.023292 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe710376-2744-4072-a57f-59582ac8d4f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:52.023900 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe710376-2744-4072-a57f-59582ac8d4f0 service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:52.024182 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe710376-2744-4072-a57f-59582ac8d4f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:52.024455 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe710376-2744-4072-a57f-59582ac8d4f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:52.026220 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-fe710376-2744-4072-a57f-59582ac8d4f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:52.040567 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe710376-2744-4072-a57f-59582ac8d4f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:52.043064 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe710376-2744-4072-a57f-59582ac8d4f0 service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:52.054233 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1140/2279] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:18:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:18:52.319014 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c16ae-2d8d-474b-8bdf-03e2423a8134 service placement] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:52.389703 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c16ae-2d8d-474b-8bdf-03e2423a8134 service placement] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:52.397816 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b7dae-0420-49f8-903f-d9530627c990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:52.398446 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b7dae-0420-49f8-903f-d9530627c990 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:52.400593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b7dae-0420-49f8-903f-d9530627c990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:52.400912 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b7dae-0420-49f8-903f-d9530627c990 admin admin] PATH_INFO: `/v3/users/e6555c0ef9444e40a87b6f2351b408c1` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:52.402057 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886b7dae-0420-49f8-903f-d9530627c990 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6555c0ef9444e40a87b6f2351b408c1)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:52.404093 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886b7dae-0420-49f8-903f-d9530627c990 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:52.451628 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-886b7dae-0420-49f8-903f-d9530627c990 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6555c0ef9444e40a87b6f2351b408c1'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:52.459758 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-886b7dae-0420-49f8-903f-d9530627c990 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6555c0ef9444e40a87b6f2351b408c1'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:52.460374 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-886b7dae-0420-49f8-903f-d9530627c990 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6555c0ef9444e40a87b6f2351b408c1'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:52.460999 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-886b7dae-0420-49f8-903f-d9530627c990 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6555c0ef9444e40a87b6f2351b408c1'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:52.468540 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-886b7dae-0420-49f8-903f-d9530627c990 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6555c0ef9444e40a87b6f2351b408c1'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:52.486056 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1140/2280] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:18:52 2022] DELETE /identity/v3/users/e6555c0ef9444e40a87b6f2351b408c1 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:18:52.490556 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f3036-a72c-4df2-be8f-7547e876e913 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:52.490862 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f3036-a72c-4df2-be8f-7547e876e913 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:52.491145 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f3036-a72c-4df2-be8f-7547e876e913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:52.510305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-bb7f3036-a72c-4df2-be8f-7547e876e913 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:18:52.511575 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-bb7f3036-a72c-4df2-be8f-7547e876e913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:52.569289 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1141/2281] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:18:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:18:52.578865 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b7dae-0420-49f8-903f-d9530627c990 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:52.580346 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-886b7dae-0420-49f8-903f-d9530627c990 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:52.600870 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:18:52.643268 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b7dae-0420-49f8-903f-d9530627c990 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:52.647010 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0406f1b-35d1-4897-9689-575a7dd209f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:52.647608 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0406f1b-35d1-4897-9689-575a7dd209f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:52.647881 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0406f1b-35d1-4897-9689-575a7dd209f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:52.648148 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0406f1b-35d1-4897-9689-575a7dd209f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:52.657711 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0406f1b-35d1-4897-9689-575a7dd209f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:52.659138 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0406f1b-35d1-4897-9689-575a7dd209f2 service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:52.668321 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1141/2282] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:18:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:18:52.931092 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f3036-a72c-4df2-be8f-7547e876e913 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:52.932532 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-bb7f3036-a72c-4df2-be8f-7547e876e913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:52.949030 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f3036-a72c-4df2-be8f-7547e876e913 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:52.954875 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34c52e0-dbd2-4337-b504-12d602c917cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:52.955598 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c52e0-dbd2-4337-b504-12d602c917cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:52.955881 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c52e0-dbd2-4337-b504-12d602c917cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:52.956223 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c52e0-dbd2-4337-b504-12d602c917cf admin admin] PATH_INFO: `/v3/projects/7983920985ad438a8725654371ffbe4f` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:52.957311 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34c52e0-dbd2-4337-b504-12d602c917cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=7983920985ad438a8725654371ffbe4f)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:52.959231 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34c52e0-dbd2-4337-b504-12d602c917cf admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:53.010145 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d34c52e0-dbd2-4337-b504-12d602c917cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7983920985ad438a8725654371ffbe4f', 'request_id': 'req-d34c52e0-dbd2-4337-b504-12d602c917cf'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:53.018781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d34c52e0-dbd2-4337-b504-12d602c917cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7983920985ad438a8725654371ffbe4f', 'request_id': 'req-d34c52e0-dbd2-4337-b504-12d602c917cf'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:53.020869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:18:53.026538 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d34c52e0-dbd2-4337-b504-12d602c917cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7983920985ad438a8725654371ffbe4f', 'request_id': 'req-d34c52e0-dbd2-4337-b504-12d602c917cf'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:53.039049 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d34c52e0-dbd2-4337-b504-12d602c917cf admin admin] The token cache is being invalidate because project 7983920985ad438a8725654371ffbe4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:18:53.039235 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d34c52e0-dbd2-4337-b504-12d602c917cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:53.039878 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d34c52e0-dbd2-4337-b504-12d602c917cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:53.041596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1142/2283] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:18:52 2022] DELETE /identity/v3/projects/7983920985ad438a8725654371ffbe4f => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:18:53.046458 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0bb5d1-cea7-44b6-ae53-943cd89ffa75 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:53.046748 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0bb5d1-cea7-44b6-ae53-943cd89ffa75 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:53.047047 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0bb5d1-cea7-44b6-ae53-943cd89ffa75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:53.067506 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-5c0bb5d1-cea7-44b6-ae53-943cd89ffa75 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:18:53.068713 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-5c0bb5d1-cea7-44b6-ae53-943cd89ffa75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:53.127845 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1142/2284] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:18:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:18:53.135555 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25775666-d26b-4c35-8893-a9fedb77fd99 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:53.135840 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25775666-d26b-4c35-8893-a9fedb77fd99 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:53.136139 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25775666-d26b-4c35-8893-a9fedb77fd99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:53.155896 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-25775666-d26b-4c35-8893-a9fedb77fd99 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:18:53.157094 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-25775666-d26b-4c35-8893-a9fedb77fd99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:53.164840 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1143/2285] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:18:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:18:53.168965 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0bb5d1-cea7-44b6-ae53-943cd89ffa75 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:53.178894 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0bb5d1-cea7-44b6-ae53-943cd89ffa75 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:53.183356 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304bbcf6-88f1-4d35-867e-c1b10b0f5e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:53.183943 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bbcf6-88f1-4d35-867e-c1b10b0f5e98 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:53.184215 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bbcf6-88f1-4d35-867e-c1b10b0f5e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:53.184483 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bbcf6-88f1-4d35-867e-c1b10b0f5e98 admin admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:53.185118 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304bbcf6-88f1-4d35-867e-c1b10b0f5e98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:18:53.185415 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304bbcf6-88f1-4d35-867e-c1b10b0f5e98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:53.187284 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304bbcf6-88f1-4d35-867e-c1b10b0f5e98 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:53.199151 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1143/2286] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:18:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:18:53.203340 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25775666-d26b-4c35-8893-a9fedb77fd99 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:53.212419 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25775666-d26b-4c35-8893-a9fedb77fd99 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:53.216988 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6929c56-44d2-4de3-9ac8-d5f8dd4c6881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:53.217552 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6929c56-44d2-4de3-9ac8-d5f8dd4c6881 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:53.217827 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6929c56-44d2-4de3-9ac8-d5f8dd4c6881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:53.218096 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6929c56-44d2-4de3-9ac8-d5f8dd4c6881 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:53.218722 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6929c56-44d2-4de3-9ac8-d5f8dd4c6881 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:53.220964 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6929c56-44d2-4de3-9ac8-d5f8dd4c6881 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:53.247916 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1144/2287] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:18:53 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:18:53.252550 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304bbcf6-88f1-4d35-867e-c1b10b0f5e98 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:53.262250 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304bbcf6-88f1-4d35-867e-c1b10b0f5e98 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:53.316364 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402fa6b5-45a0-44ce-809b-1981817d815b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:53.316955 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402fa6b5-45a0-44ce-809b-1981817d815b admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:53.317228 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402fa6b5-45a0-44ce-809b-1981817d815b admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:53.317494 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402fa6b5-45a0-44ce-809b-1981817d815b admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:53.318118 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402fa6b5-45a0-44ce-809b-1981817d815b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:53.320025 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402fa6b5-45a0-44ce-809b-1981817d815b admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:53.342026 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1144/2288] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:18:53 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:18:53.346200 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6929c56-44d2-4de3-9ac8-d5f8dd4c6881 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:53.356592 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6929c56-44d2-4de3-9ac8-d5f8dd4c6881 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:53.361791 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9295c-10be-47a2-a841-35927cfaf397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:53.362415 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9295c-10be-47a2-a841-35927cfaf397 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:53.362802 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9295c-10be-47a2-a841-35927cfaf397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:53.363105 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9295c-10be-47a2-a841-35927cfaf397 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:53.363741 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a9295c-10be-47a2-a841-35927cfaf397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:53.365302 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a9295c-10be-47a2-a841-35927cfaf397 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:53.375047 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1145/2289] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:18:53.379450 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402fa6b5-45a0-44ce-809b-1981817d815b admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:53.389397 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402fa6b5-45a0-44ce-809b-1981817d815b admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:53.394036 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6785347-60bc-4f42-9f20-5cf135b1fbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:53.394623 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6785347-60bc-4f42-9f20-5cf135b1fbde admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:53.394937 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6785347-60bc-4f42-9f20-5cf135b1fbde admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:53.395210 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6785347-60bc-4f42-9f20-5cf135b1fbde admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:53.395845 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6785347-60bc-4f42-9f20-5cf135b1fbde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:53.397416 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6785347-60bc-4f42-9f20-5cf135b1fbde admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:53.408825 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1145/2290] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:18:53.412716 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9295c-10be-47a2-a841-35927cfaf397 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:53.423138 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9295c-10be-47a2-a841-35927cfaf397 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:53.427667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b61be9-c851-4067-a010-52f3af584830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:53.428297 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b61be9-c851-4067-a010-52f3af584830 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:53.428571 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b61be9-c851-4067-a010-52f3af584830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:53.428838 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b61be9-c851-4067-a010-52f3af584830 admin admin] PATH_INFO: `/v3/projects/76df052793e8406a93615d102732f40b/users/648312b06d5045148fd181eedd2bd8de/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:53.443796 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b61be9-c851-4067-a010-52f3af584830 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76df052793e8406a93615d102732f40b, user_id=648312b06d5045148fd181eedd2bd8de, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:53.446382 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b61be9-c851-4067-a010-52f3af584830 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:53.459433 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1146/2291] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:18:53 2022] PUT /identity/v3/projects/76df052793e8406a93615d102732f40b/users/648312b06d5045148fd181eedd2bd8de/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:18:53.469806 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50212667-de0b-49f4-b395-98f023261879 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:53.470088 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50212667-de0b-49f4-b395-98f023261879 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:53.470385 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50212667-de0b-49f4-b395-98f023261879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:53.498339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-50212667-de0b-49f4-b395-98f023261879 None None] MFA Rules not processed for user `648312b06d5045148fd181eedd2bd8de`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:18:53.499574 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-50212667-de0b-49f4-b395-98f023261879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:53.511768 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:18:53.567023 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1146/2292] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:18:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:18:53.571476 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b61be9-c851-4067-a010-52f3af584830 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:53.636705 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b61be9-c851-4067-a010-52f3af584830 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:53.641430 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618ad3c-bc7b-48e6-bdb0-ba5b3bbf042f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:53.642034 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618ad3c-bc7b-48e6-bdb0-ba5b3bbf042f admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:53.642315 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618ad3c-bc7b-48e6-bdb0-ba5b3bbf042f admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:53.642589 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618ad3c-bc7b-48e6-bdb0-ba5b3bbf042f admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:53.643260 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5618ad3c-bc7b-48e6-bdb0-ba5b3bbf042f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:53.645140 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5618ad3c-bc7b-48e6-bdb0-ba5b3bbf042f admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:53.670953 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1147/2293] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:18:53 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:18:53.675577 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50212667-de0b-49f4-b395-98f023261879 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:53.686039 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50212667-de0b-49f4-b395-98f023261879 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:53.741883 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c170a4f2-d99d-49f5-a86b-ebaa304e33bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:53.742488 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170a4f2-d99d-49f5-a86b-ebaa304e33bd admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:53.742769 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170a4f2-d99d-49f5-a86b-ebaa304e33bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:53.743093 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170a4f2-d99d-49f5-a86b-ebaa304e33bd admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:53.743726 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c170a4f2-d99d-49f5-a86b-ebaa304e33bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:53.745631 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c170a4f2-d99d-49f5-a86b-ebaa304e33bd admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:53.765502 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1147/2294] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:18:53 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:18:53.769620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618ad3c-bc7b-48e6-bdb0-ba5b3bbf042f admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:53.779713 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618ad3c-bc7b-48e6-bdb0-ba5b3bbf042f admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:53.784809 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e752c2-9b09-4020-a5f1-374a18e8a63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:53.785406 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e752c2-9b09-4020-a5f1-374a18e8a63f admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:53.785685 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e752c2-9b09-4020-a5f1-374a18e8a63f admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:53.785958 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e752c2-9b09-4020-a5f1-374a18e8a63f admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:53.786600 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e752c2-9b09-4020-a5f1-374a18e8a63f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:53.788207 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e752c2-9b09-4020-a5f1-374a18e8a63f admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:53.799156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1148/2295] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:18:53.803100 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c170a4f2-d99d-49f5-a86b-ebaa304e33bd admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:53.813076 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c170a4f2-d99d-49f5-a86b-ebaa304e33bd admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:53.817657 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce918f9-e76b-43f7-9922-ea6efb93967d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:53.818234 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce918f9-e76b-43f7-9922-ea6efb93967d admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:53.818508 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce918f9-e76b-43f7-9922-ea6efb93967d admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:53.818781 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce918f9-e76b-43f7-9922-ea6efb93967d admin admin] PATH_INFO: `/v3/projects/455841c4e5bf49ef95dd5f72c369fc8e/users/e7cf0c7935234297943a00f5417f8f3e/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:53.833780 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce918f9-e76b-43f7-9922-ea6efb93967d admin admin] RBAC: Authorizing `identity:create_grant(project_id=455841c4e5bf49ef95dd5f72c369fc8e, user_id=e7cf0c7935234297943a00f5417f8f3e, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:53.836624 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce918f9-e76b-43f7-9922-ea6efb93967d admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:53.851259 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1148/2296] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:18:53 2022] PUT /identity/v3/projects/455841c4e5bf49ef95dd5f72c369fc8e/users/e7cf0c7935234297943a00f5417f8f3e/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:18:53.855531 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e752c2-9b09-4020-a5f1-374a18e8a63f admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:53.865942 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e752c2-9b09-4020-a5f1-374a18e8a63f admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:53.916380 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:18:53.924230 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610d6cc-4ddb-4282-8255-06ccecde9f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:53.924915 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610d6cc-4ddb-4282-8255-06ccecde9f0d admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:53.925294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610d6cc-4ddb-4282-8255-06ccecde9f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:53.925942 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610d6cc-4ddb-4282-8255-06ccecde9f0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:53.926588 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610d6cc-4ddb-4282-8255-06ccecde9f0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:53.928491 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610d6cc-4ddb-4282-8255-06ccecde9f0d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:53.939863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1149/2297] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:18:53.944059 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce918f9-e76b-43f7-9922-ea6efb93967d admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:53.955231 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce918f9-e76b-43f7-9922-ea6efb93967d admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:53.959909 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf730a-8816-4845-a71d-0efc0e53f7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:53.960539 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf730a-8816-4845-a71d-0efc0e53f7bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:53.960935 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf730a-8816-4845-a71d-0efc0e53f7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:53.961217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf730a-8816-4845-a71d-0efc0e53f7bc admin admin] PATH_INFO: `/v3/projects/455841c4e5bf49ef95dd5f72c369fc8e/users/e7cf0c7935234297943a00f5417f8f3e/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:53.963162 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf730a-8816-4845-a71d-0efc0e53f7bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=455841c4e5bf49ef95dd5f72c369fc8e, user_id=e7cf0c7935234297943a00f5417f8f3e, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:53.965783 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf730a-8816-4845-a71d-0efc0e53f7bc admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:53.976436 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1149/2298] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:18:53 2022] PUT /identity/v3/projects/455841c4e5bf49ef95dd5f72c369fc8e/users/e7cf0c7935234297943a00f5417f8f3e/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:18:53.987962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7a28e-1646-4fc3-9ff1-630746924ef2 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:53.988249 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7a28e-1646-4fc3-9ff1-630746924ef2 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:53.988528 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7a28e-1646-4fc3-9ff1-630746924ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:54.018355 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-70d7a28e-1646-4fc3-9ff1-630746924ef2 None None] MFA Rules not processed for user `e7cf0c7935234297943a00f5417f8f3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:18:54.019729 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-70d7a28e-1646-4fc3-9ff1-630746924ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:54.088596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1150/2299] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:18:53 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:18:54.100489 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf730a-8816-4845-a71d-0efc0e53f7bc admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:54.102202 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-5abf730a-8816-4845-a71d-0efc0e53f7bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:54.160932 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf730a-8816-4845-a71d-0efc0e53f7bc admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:54.164750 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fdc856-ab6b-4407-b4a7-f9c2f46a51bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:54.165362 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fdc856-ab6b-4407-b4a7-f9c2f46a51bb service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:54.165646 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fdc856-ab6b-4407-b4a7-f9c2f46a51bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:54.165916 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fdc856-ab6b-4407-b4a7-f9c2f46a51bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:54.175369 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fdc856-ab6b-4407-b4a7-f9c2f46a51bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:54.176809 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fdc856-ab6b-4407-b4a7-f9c2f46a51bb service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:54.186746 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1150/2300] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:18:54 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:18:54.574072 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d7a28e-1646-4fc3-9ff1-630746924ef2 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:54.634785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d7a28e-1646-4fc3-9ff1-630746924ef2 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:54.639394 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce2990-3a99-4fc6-ad41-2a8ea75e6d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:54.640027 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce2990-3a99-4fc6-ad41-2a8ea75e6d04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:54.640305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce2990-3a99-4fc6-ad41-2a8ea75e6d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:54.640596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce2990-3a99-4fc6-ad41-2a8ea75e6d04 admin admin] PATH_INFO: `/v3/users/648312b06d5045148fd181eedd2bd8de` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:54.642335 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ce2990-3a99-4fc6-ad41-2a8ea75e6d04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=648312b06d5045148fd181eedd2bd8de)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:54.644258 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ce2990-3a99-4fc6-ad41-2a8ea75e6d04 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:54.693855 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f3ce2990-3a99-4fc6-ad41-2a8ea75e6d04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '648312b06d5045148fd181eedd2bd8de'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:54.701411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f3ce2990-3a99-4fc6-ad41-2a8ea75e6d04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '648312b06d5045148fd181eedd2bd8de'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:54.702180 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f3ce2990-3a99-4fc6-ad41-2a8ea75e6d04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '648312b06d5045148fd181eedd2bd8de'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:54.702825 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f3ce2990-3a99-4fc6-ad41-2a8ea75e6d04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '648312b06d5045148fd181eedd2bd8de'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:54.710039 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f3ce2990-3a99-4fc6-ad41-2a8ea75e6d04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '648312b06d5045148fd181eedd2bd8de'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:54.722426 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:18:54.730770 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1151/2301] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:18:54 2022] DELETE /identity/v3/users/648312b06d5045148fd181eedd2bd8de => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:18:54.734167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933744c-4b2d-49b0-b582-c27a59d2353e None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:54.734666 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933744c-4b2d-49b0-b582-c27a59d2353e None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:54.735309 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933744c-4b2d-49b0-b582-c27a59d2353e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:54.754732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-1933744c-4b2d-49b0-b582-c27a59d2353e None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:18:54.755944 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-1933744c-4b2d-49b0-b582-c27a59d2353e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:54.759910 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:18:54.824628 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1151/2302] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:18:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:18:54.834587 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce2990-3a99-4fc6-ad41-2a8ea75e6d04 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:54.836076 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f3ce2990-3a99-4fc6-ad41-2a8ea75e6d04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:54.895549 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce2990-3a99-4fc6-ad41-2a8ea75e6d04 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:54.899490 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8a104-b90f-41f8-b2d0-f58b6d490d85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:54.900103 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8a104-b90f-41f8-b2d0-f58b6d490d85 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:54.900383 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8a104-b90f-41f8-b2d0-f58b6d490d85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:54.900656 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8a104-b90f-41f8-b2d0-f58b6d490d85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:54.909953 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8a104-b90f-41f8-b2d0-f58b6d490d85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:54.911422 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8a104-b90f-41f8-b2d0-f58b6d490d85 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:54.921294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1152/2303] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:18:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:18:55.497432 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1933744c-4b2d-49b0-b582-c27a59d2353e None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:55.499021 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-1933744c-4b2d-49b0-b582-c27a59d2353e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:55.510994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1933744c-4b2d-49b0-b582-c27a59d2353e None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:55.518339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051788db-d383-40c1-b72b-60d3737d0308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:55.519270 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051788db-d383-40c1-b72b-60d3737d0308 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:55.519384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051788db-d383-40c1-b72b-60d3737d0308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:55.519624 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051788db-d383-40c1-b72b-60d3737d0308 admin admin] PATH_INFO: `/v3/projects/76df052793e8406a93615d102732f40b` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:55.520869 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051788db-d383-40c1-b72b-60d3737d0308 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76df052793e8406a93615d102732f40b)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:55.522813 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051788db-d383-40c1-b72b-60d3737d0308 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:55.580019 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-051788db-d383-40c1-b72b-60d3737d0308 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76df052793e8406a93615d102732f40b', 'request_id': 'req-051788db-d383-40c1-b72b-60d3737d0308'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:55.585740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-051788db-d383-40c1-b72b-60d3737d0308 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76df052793e8406a93615d102732f40b', 'request_id': 'req-051788db-d383-40c1-b72b-60d3737d0308'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:55.593521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-051788db-d383-40c1-b72b-60d3737d0308 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76df052793e8406a93615d102732f40b', 'request_id': 'req-051788db-d383-40c1-b72b-60d3737d0308'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:55.608615 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-051788db-d383-40c1-b72b-60d3737d0308 admin admin] The token cache is being invalidate because project 76df052793e8406a93615d102732f40b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:18:55.609037 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-051788db-d383-40c1-b72b-60d3737d0308 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:55.609813 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-051788db-d383-40c1-b72b-60d3737d0308 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:55.612417 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1152/2304] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:18:55 2022] DELETE /identity/v3/projects/76df052793e8406a93615d102732f40b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:18:55.616316 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8a104-b90f-41f8-b2d0-f58b6d490d85 service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:55.626186 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8a104-b90f-41f8-b2d0-f58b6d490d85 service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:55.698819 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e30655-f58a-40f9-b07a-14e87f305bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:55.699667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e30655-f58a-40f9-b07a-14e87f305bb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:55.700524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e30655-f58a-40f9-b07a-14e87f305bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:55.700870 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e30655-f58a-40f9-b07a-14e87f305bb4 admin admin] PATH_INFO: `/v3/users/e7cf0c7935234297943a00f5417f8f3e` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:55.704576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e30655-f58a-40f9-b07a-14e87f305bb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7cf0c7935234297943a00f5417f8f3e)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:55.707422 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e30655-f58a-40f9-b07a-14e87f305bb4 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:55.785868 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a9e30655-f58a-40f9-b07a-14e87f305bb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7cf0c7935234297943a00f5417f8f3e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:55.804704 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a9e30655-f58a-40f9-b07a-14e87f305bb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7cf0c7935234297943a00f5417f8f3e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:55.805399 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a9e30655-f58a-40f9-b07a-14e87f305bb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7cf0c7935234297943a00f5417f8f3e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:55.806102 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a9e30655-f58a-40f9-b07a-14e87f305bb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7cf0c7935234297943a00f5417f8f3e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:55.826121 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a9e30655-f58a-40f9-b07a-14e87f305bb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7cf0c7935234297943a00f5417f8f3e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:55.853008 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1153/2305] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:18:55 2022] DELETE /identity/v3/users/e7cf0c7935234297943a00f5417f8f3e => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:18:56.181928 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051788db-d383-40c1-b72b-60d3737d0308 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:56.183597 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-051788db-d383-40c1-b72b-60d3737d0308 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:56.198167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051788db-d383-40c1-b72b-60d3737d0308 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:56.260672 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6520f2a9-b0e7-4e44-9933-3878a10041aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:56.261321 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520f2a9-b0e7-4e44-9933-3878a10041aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:56.261602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520f2a9-b0e7-4e44-9933-3878a10041aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:56.261874 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520f2a9-b0e7-4e44-9933-3878a10041aa admin admin] PATH_INFO: `/v3/projects/455841c4e5bf49ef95dd5f72c369fc8e` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:56.263072 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6520f2a9-b0e7-4e44-9933-3878a10041aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=455841c4e5bf49ef95dd5f72c369fc8e)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:56.265014 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6520f2a9-b0e7-4e44-9933-3878a10041aa admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:56.309925 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:18:56.328132 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-6520f2a9-b0e7-4e44-9933-3878a10041aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '455841c4e5bf49ef95dd5f72c369fc8e', 'request_id': 'req-6520f2a9-b0e7-4e44-9933-3878a10041aa'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:56.334623 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-6520f2a9-b0e7-4e44-9933-3878a10041aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '455841c4e5bf49ef95dd5f72c369fc8e', 'request_id': 'req-6520f2a9-b0e7-4e44-9933-3878a10041aa'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:56.342410 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-6520f2a9-b0e7-4e44-9933-3878a10041aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '455841c4e5bf49ef95dd5f72c369fc8e', 'request_id': 'req-6520f2a9-b0e7-4e44-9933-3878a10041aa'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:56.356934 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-6520f2a9-b0e7-4e44-9933-3878a10041aa admin admin] The token cache is being invalidate because project 455841c4e5bf49ef95dd5f72c369fc8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:18:56.357248 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-6520f2a9-b0e7-4e44-9933-3878a10041aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:56.357891 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-6520f2a9-b0e7-4e44-9933-3878a10041aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:56.360172 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1153/2306] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:18:56 2022] DELETE /identity/v3/projects/455841c4e5bf49ef95dd5f72c369fc8e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:18:56.365452 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326d96c-0938-46ed-afe1-bc38ec627f40 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:56.365850 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326d96c-0938-46ed-afe1-bc38ec627f40 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:56.366135 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326d96c-0938-46ed-afe1-bc38ec627f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:56.386734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-7326d96c-0938-46ed-afe1-bc38ec627f40 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:18:56.388027 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7326d96c-0938-46ed-afe1-bc38ec627f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:56.401880 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:18:56.451755 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1154/2307] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:18:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:18:56.460746 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2f8f5-588a-4327-872a-e030ea1b73af None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:56.460869 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2f8f5-588a-4327-872a-e030ea1b73af None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:56.461078 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2f8f5-588a-4327-872a-e030ea1b73af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:56.490522 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-2bd2f8f5-588a-4327-872a-e030ea1b73af None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:18:56.491828 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-2bd2f8f5-588a-4327-872a-e030ea1b73af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:56.499839 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1154/2308] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:18:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:18:56.504840 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7326d96c-0938-46ed-afe1-bc38ec627f40 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:56.514472 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7326d96c-0938-46ed-afe1-bc38ec627f40 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:56.519108 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e524dc-8bf0-411c-8420-609aac3f08ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:56.519715 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e524dc-8bf0-411c-8420-609aac3f08ea admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:56.520000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e524dc-8bf0-411c-8420-609aac3f08ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:56.520271 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e524dc-8bf0-411c-8420-609aac3f08ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:56.520923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e524dc-8bf0-411c-8420-609aac3f08ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:18:56.521223 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e524dc-8bf0-411c-8420-609aac3f08ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:56.522806 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e524dc-8bf0-411c-8420-609aac3f08ea admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:56.534539 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1155/2309] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:18:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:18:56.538799 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2f8f5-588a-4327-872a-e030ea1b73af None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:56.548439 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2f8f5-588a-4327-872a-e030ea1b73af None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:56.553022 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e92b856-e544-4d56-a137-e7a589b000ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:56.553613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e92b856-e544-4d56-a137-e7a589b000ff admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:56.553900 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e92b856-e544-4d56-a137-e7a589b000ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:56.554144 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e92b856-e544-4d56-a137-e7a589b000ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:56.554764 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e92b856-e544-4d56-a137-e7a589b000ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:56.556671 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e92b856-e544-4d56-a137-e7a589b000ff admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:56.584271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1155/2310] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:18:56 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:18:56.588435 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e524dc-8bf0-411c-8420-609aac3f08ea admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:56.600597 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e524dc-8bf0-411c-8420-609aac3f08ea admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:56.662633 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d4dad7-f306-453e-8caf-fd8124869f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:56.663331 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4dad7-f306-453e-8caf-fd8124869f09 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:56.663542 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4dad7-f306-453e-8caf-fd8124869f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:56.663812 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4dad7-f306-453e-8caf-fd8124869f09 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:56.664452 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d4dad7-f306-453e-8caf-fd8124869f09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:56.666346 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d4dad7-f306-453e-8caf-fd8124869f09 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:56.688083 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1156/2311] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:18:56 2022] POST /identity/v3/users => generated 283 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:18:56.692263 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e92b856-e544-4d56-a137-e7a589b000ff admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:56.702299 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e92b856-e544-4d56-a137-e7a589b000ff admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:56.707514 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570b40b-e65e-4dde-9ce9-c2f236dc191a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:56.708100 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570b40b-e65e-4dde-9ce9-c2f236dc191a admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:56.708377 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570b40b-e65e-4dde-9ce9-c2f236dc191a admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:56.708645 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570b40b-e65e-4dde-9ce9-c2f236dc191a admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:56.709305 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a570b40b-e65e-4dde-9ce9-c2f236dc191a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:56.710905 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a570b40b-e65e-4dde-9ce9-c2f236dc191a admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:56.721657 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1156/2312] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:18:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:18:56.725469 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d4dad7-f306-453e-8caf-fd8124869f09 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:56.736137 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d4dad7-f306-453e-8caf-fd8124869f09 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:56.741042 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ffa9ce-9aa3-4f70-a741-5a4f4d10bb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:56.741643 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ffa9ce-9aa3-4f70-a741-5a4f4d10bb78 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:56.741924 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ffa9ce-9aa3-4f70-a741-5a4f4d10bb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:56.742217 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ffa9ce-9aa3-4f70-a741-5a4f4d10bb78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:56.742921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ffa9ce-9aa3-4f70-a741-5a4f4d10bb78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:56.744541 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ffa9ce-9aa3-4f70-a741-5a4f4d10bb78 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:56.755095 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1157/2313] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:18:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:18:56.759873 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570b40b-e65e-4dde-9ce9-c2f236dc191a admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:56.769809 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570b40b-e65e-4dde-9ce9-c2f236dc191a admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:56.775166 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f77293-2b95-4e32-9b12-555c5fb41ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:56.775972 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f77293-2b95-4e32-9b12-555c5fb41ce1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:56.776254 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f77293-2b95-4e32-9b12-555c5fb41ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:56.776587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f77293-2b95-4e32-9b12-555c5fb41ce1 admin admin] PATH_INFO: `/v3/projects/1839505f4c284acea50dc0dcc1127a91/users/ad34a63c233d481d866446df065b19ce/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:56.792032 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f77293-2b95-4e32-9b12-555c5fb41ce1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1839505f4c284acea50dc0dcc1127a91, user_id=ad34a63c233d481d866446df065b19ce, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:56.794765 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f77293-2b95-4e32-9b12-555c5fb41ce1 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:56.808776 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1157/2314] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:18:56 2022] PUT /identity/v3/projects/1839505f4c284acea50dc0dcc1127a91/users/ad34a63c233d481d866446df065b19ce/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:18:56.819287 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccf113-bdbc-4431-8347-c36253eb0ea1 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:56.819579 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccf113-bdbc-4431-8347-c36253eb0ea1 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:56.819855 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccf113-bdbc-4431-8347-c36253eb0ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:56.849833 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-31ccf113-bdbc-4431-8347-c36253eb0ea1 None None] MFA Rules not processed for user `ad34a63c233d481d866446df065b19ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:18:56.851077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-31ccf113-bdbc-4431-8347-c36253eb0ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:56.912496 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1158/2315] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:18:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:18:56.917163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f77293-2b95-4e32-9b12-555c5fb41ce1 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:56.977819 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f77293-2b95-4e32-9b12-555c5fb41ce1 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:56.982362 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab29a4-573a-4120-9bc0-062a250dddbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:56.982988 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab29a4-573a-4120-9bc0-062a250dddbb admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:56.983261 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab29a4-573a-4120-9bc0-062a250dddbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:56.983523 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab29a4-573a-4120-9bc0-062a250dddbb admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:56.984148 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab29a4-573a-4120-9bc0-062a250dddbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:56.986005 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab29a4-573a-4120-9bc0-062a250dddbb admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:57.014967 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1158/2316] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:18:56 2022] POST /identity/v3/projects => generated 365 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:18:57.023240 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ccf113-bdbc-4431-8347-c36253eb0ea1 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:57.035658 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ccf113-bdbc-4431-8347-c36253eb0ea1 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:57.048488 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:18:57.076503 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d18663-f558-4c3d-b244-6d9caeed09a2 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:57.076784 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d18663-f558-4c3d-b244-6d9caeed09a2 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:57.077059 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d18663-f558-4c3d-b244-6d9caeed09a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:57.096968 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60bac63-d815-4e80-884a-16e0858ef4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:57.097573 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60bac63-d815-4e80-884a-16e0858ef4b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:57.097829 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-89d18663-f558-4c3d-b244-6d9caeed09a2 None None] MFA Rules not processed for user `1405e758a3af4e8ba659f498cd28ace1`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:18:57.097942 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60bac63-d815-4e80-884a-16e0858ef4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:57.098099 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60bac63-d815-4e80-884a-16e0858ef4b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:57.098720 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60bac63-d815-4e80-884a-16e0858ef4b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:57.099063 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-89d18663-f558-4c3d-b244-6d9caeed09a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:57.100917 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60bac63-d815-4e80-884a-16e0858ef4b3 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:57.128932 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1159/2317] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:18:57 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:18:57.129964 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60bac63-d815-4e80-884a-16e0858ef4b3 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:57.139622 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60bac63-d815-4e80-884a-16e0858ef4b3 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:57.144349 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:18:57.145094 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b3b5b-6c6a-4b7f-8a7b-15b68e1e23e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:57.145672 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b3b5b-6c6a-4b7f-8a7b-15b68e1e23e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:57.145953 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b3b5b-6c6a-4b7f-8a7b-15b68e1e23e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:57.146286 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b3b5b-6c6a-4b7f-8a7b-15b68e1e23e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:57.147115 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062b3b5b-6c6a-4b7f-8a7b-15b68e1e23e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:57.148756 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062b3b5b-6c6a-4b7f-8a7b-15b68e1e23e8 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:57.163358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1160/2318] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:18:57.163548 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1159/2319] 10.222.0.42 () {58 vars in 967 bytes} [Mon Sep 12 14:18:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:18:57.164777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b3b5b-6c6a-4b7f-8a7b-15b68e1e23e8 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:57.175066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b3b5b-6c6a-4b7f-8a7b-15b68e1e23e8 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:57.179954 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a8384-3c7a-4b10-b276-ad52ddc7c560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:57.180499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a8384-3c7a-4b10-b276-ad52ddc7c560 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:57.180778 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a8384-3c7a-4b10-b276-ad52ddc7c560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:57.181074 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a8384-3c7a-4b10-b276-ad52ddc7c560 admin admin] PATH_INFO: `/v3/projects/c254427d1fba47f3849c72342b61a4d5/users/36bffe0bbf5f40dcbafd8ee45f0b4af7/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:57.196980 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476a8384-3c7a-4b10-b276-ad52ddc7c560 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c254427d1fba47f3849c72342b61a4d5, user_id=36bffe0bbf5f40dcbafd8ee45f0b4af7, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:57.199713 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476a8384-3c7a-4b10-b276-ad52ddc7c560 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:57.210615 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d18663-f558-4c3d-b244-6d9caeed09a2 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:57.212365 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-89d18663-f558-4c3d-b244-6d9caeed09a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:57.214345 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1161/2320] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:18:57 2022] PUT /identity/v3/projects/c254427d1fba47f3849c72342b61a4d5/users/36bffe0bbf5f40dcbafd8ee45f0b4af7/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:18:57.218038 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a8384-3c7a-4b10-b276-ad52ddc7c560 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:57.279341 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a8384-3c7a-4b10-b276-ad52ddc7c560 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:57.283354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d18663-f558-4c3d-b244-6d9caeed09a2 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:57.284528 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d262a57-5875-4dce-8326-c32cca8e0d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:57.285124 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d262a57-5875-4dce-8326-c32cca8e0d2b admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:57.285403 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d262a57-5875-4dce-8326-c32cca8e0d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:57.285676 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d262a57-5875-4dce-8326-c32cca8e0d2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:57.286318 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d262a57-5875-4dce-8326-c32cca8e0d2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:57.287910 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d262a57-5875-4dce-8326-c32cca8e0d2b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:57.299133 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1162/2321] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:18:57.303431 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d262a57-5875-4dce-8326-c32cca8e0d2b admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:57.314456 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d262a57-5875-4dce-8326-c32cca8e0d2b admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:57.319228 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5480c8da-ddf3-4e47-994b-55fcc461bf9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:57.319859 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480c8da-ddf3-4e47-994b-55fcc461bf9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:57.320142 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480c8da-ddf3-4e47-994b-55fcc461bf9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:57.320412 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480c8da-ddf3-4e47-994b-55fcc461bf9c admin admin] PATH_INFO: `/v3/projects/c254427d1fba47f3849c72342b61a4d5/users/36bffe0bbf5f40dcbafd8ee45f0b4af7/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:57.322295 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5480c8da-ddf3-4e47-994b-55fcc461bf9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c254427d1fba47f3849c72342b61a4d5, user_id=36bffe0bbf5f40dcbafd8ee45f0b4af7, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:57.324904 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5480c8da-ddf3-4e47-994b-55fcc461bf9c admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:57.335932 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1163/2322] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:18:57 2022] PUT /identity/v3/projects/c254427d1fba47f3849c72342b61a4d5/users/36bffe0bbf5f40dcbafd8ee45f0b4af7/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:18:57.343597 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439aa6b-1b0d-4e0e-9304-4f33c8b087fb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a87ea9c3c5409b97a35c4857ae3ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:57.344258 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439aa6b-1b0d-4e0e-9304-4f33c8b087fb service swift] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:57.344667 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439aa6b-1b0d-4e0e-9304-4f33c8b087fb service swift] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:57.344943 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439aa6b-1b0d-4e0e-9304-4f33c8b087fb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:57.347273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc907200-5bc0-446e-9d4d-34243309bfac None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:57.347558 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc907200-5bc0-446e-9d4d-34243309bfac None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:57.347838 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc907200-5bc0-446e-9d4d-34243309bfac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:57.376355 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-cc907200-5bc0-446e-9d4d-34243309bfac None None] MFA Rules not processed for user `36bffe0bbf5f40dcbafd8ee45f0b4af7`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:18:57.377556 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-cc907200-5bc0-446e-9d4d-34243309bfac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:57.403203 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5439aa6b-1b0d-4e0e-9304-4f33c8b087fb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:57.404855 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5439aa6b-1b0d-4e0e-9304-4f33c8b087fb service swift] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:57.416525 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1160/2323] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Sep 12 14:18:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 12 14:18:57.425284 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:18:57.459765 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1164/2324] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:18:57 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:18:57.471263 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439aa6b-1b0d-4e0e-9304-4f33c8b087fb service swift] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:57.472643 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-5439aa6b-1b0d-4e0e-9304-4f33c8b087fb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:57.509389 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:18:57.533977 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439aa6b-1b0d-4e0e-9304-4f33c8b087fb service swift] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:57.585141 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3258e103-5687-4406-a85e-53953bbba82b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:57.585751 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3258e103-5687-4406-a85e-53953bbba82b service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:57.586034 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3258e103-5687-4406-a85e-53953bbba82b service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:57.586332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3258e103-5687-4406-a85e-53953bbba82b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:57.595509 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3258e103-5687-4406-a85e-53953bbba82b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:57.596942 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3258e103-5687-4406-a85e-53953bbba82b service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:57.606862 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1161/2325] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:18:57 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:18:58.070785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc907200-5bc0-446e-9d4d-34243309bfac None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:58.072388 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-cc907200-5bc0-446e-9d4d-34243309bfac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:58.194683 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc907200-5bc0-446e-9d4d-34243309bfac None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:58.201199 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc80a6-6eca-47d2-a453-ee0a5463276a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:58.201959 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc80a6-6eca-47d2-a453-ee0a5463276a service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:58.202726 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc80a6-6eca-47d2-a453-ee0a5463276a service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:58.203126 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc80a6-6eca-47d2-a453-ee0a5463276a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:58.205194 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-34fc80a6-6eca-47d2-a453-ee0a5463276a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:58.218123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc80a6-6eca-47d2-a453-ee0a5463276a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:58.220455 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc80a6-6eca-47d2-a453-ee0a5463276a service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:58.245041 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1165/2326] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:18:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:18:58.272276 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3258e103-5687-4406-a85e-53953bbba82b service cinder] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:58.356929 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3258e103-5687-4406-a85e-53953bbba82b service cinder] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:58.364105 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e00aa4-021e-46ac-bcb9-93afec792dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:58.364747 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e00aa4-021e-46ac-bcb9-93afec792dee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:58.365318 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e00aa4-021e-46ac-bcb9-93afec792dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:58.365318 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e00aa4-021e-46ac-bcb9-93afec792dee admin admin] PATH_INFO: `/v3/users/ad34a63c233d481d866446df065b19ce` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:58.366516 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e00aa4-021e-46ac-bcb9-93afec792dee admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad34a63c233d481d866446df065b19ce)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:58.368523 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e00aa4-021e-46ac-bcb9-93afec792dee admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:58.428412 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-d3e00aa4-021e-46ac-bcb9-93afec792dee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad34a63c233d481d866446df065b19ce'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:58.436683 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-d3e00aa4-021e-46ac-bcb9-93afec792dee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad34a63c233d481d866446df065b19ce'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:58.437384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-d3e00aa4-021e-46ac-bcb9-93afec792dee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad34a63c233d481d866446df065b19ce'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:58.438016 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-d3e00aa4-021e-46ac-bcb9-93afec792dee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad34a63c233d481d866446df065b19ce'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:58.445135 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-d3e00aa4-021e-46ac-bcb9-93afec792dee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad34a63c233d481d866446df065b19ce'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:58.466006 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1162/2327] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:18:58 2022] DELETE /identity/v3/users/ad34a63c233d481d866446df065b19ce => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:18:58.471440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49efbf31-2c48-4435-bc9b-9445cde26781 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:58.471971 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49efbf31-2c48-4435-bc9b-9445cde26781 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:58.472253 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49efbf31-2c48-4435-bc9b-9445cde26781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:58.491182 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-49efbf31-2c48-4435-bc9b-9445cde26781 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:18:58.492394 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-49efbf31-2c48-4435-bc9b-9445cde26781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:58.556331 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1166/2328] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:18:58.562686 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e00aa4-021e-46ac-bcb9-93afec792dee admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:58.566058 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-d3e00aa4-021e-46ac-bcb9-93afec792dee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:58.566205 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49efbf31-2c48-4435-bc9b-9445cde26781 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:58.567273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-49efbf31-2c48-4435-bc9b-9445cde26781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:58.579848 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:18:58.645390 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49efbf31-2c48-4435-bc9b-9445cde26781 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:58.649867 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2558e19-c06a-4e7b-b8d6-b4dde2239979 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:58.651079 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2558e19-c06a-4e7b-b8d6-b4dde2239979 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:58.651537 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2558e19-c06a-4e7b-b8d6-b4dde2239979 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:58.652077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2558e19-c06a-4e7b-b8d6-b4dde2239979 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:58.653067 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e00aa4-021e-46ac-bcb9-93afec792dee admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:58.658596 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4108a1ce-0051-4595-8676-534a05b151b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:58.659231 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4108a1ce-0051-4595-8676-534a05b151b1 service nova] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:58.659505 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4108a1ce-0051-4595-8676-534a05b151b1 service nova] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:58.659766 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4108a1ce-0051-4595-8676-534a05b151b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:58.661560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-4108a1ce-0051-4595-8676-534a05b151b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:58.667427 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2558e19-c06a-4e7b-b8d6-b4dde2239979 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:58.668899 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2558e19-c06a-4e7b-b8d6-b4dde2239979 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:58.680089 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1167/2329] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:18:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:18:58.727101 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4108a1ce-0051-4595-8676-534a05b151b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:58.728710 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4108a1ce-0051-4595-8676-534a05b151b1 service nova] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:58.740305 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1163/2330] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:18:58 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:18:59.140050 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2558e19-c06a-4e7b-b8d6-b4dde2239979 service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:59.141887 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e2558e19-c06a-4e7b-b8d6-b4dde2239979 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:59.155257 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2558e19-c06a-4e7b-b8d6-b4dde2239979 service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:59.161370 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b78d48-03de-4675-80e0-03e28cdefadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:59.162003 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b78d48-03de-4675-80e0-03e28cdefadd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:59.162297 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b78d48-03de-4675-80e0-03e28cdefadd admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:59.162568 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b78d48-03de-4675-80e0-03e28cdefadd admin admin] PATH_INFO: `/v3/projects/1839505f4c284acea50dc0dcc1127a91` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:59.163769 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b78d48-03de-4675-80e0-03e28cdefadd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1839505f4c284acea50dc0dcc1127a91)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:59.165837 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b78d48-03de-4675-80e0-03e28cdefadd admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:59.168637 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:18:59.218428 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-30b78d48-03de-4675-80e0-03e28cdefadd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1839505f4c284acea50dc0dcc1127a91', 'request_id': 'req-30b78d48-03de-4675-80e0-03e28cdefadd'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:59.228096 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-30b78d48-03de-4675-80e0-03e28cdefadd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1839505f4c284acea50dc0dcc1127a91', 'request_id': 'req-30b78d48-03de-4675-80e0-03e28cdefadd'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:59.234328 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-30b78d48-03de-4675-80e0-03e28cdefadd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1839505f4c284acea50dc0dcc1127a91', 'request_id': 'req-30b78d48-03de-4675-80e0-03e28cdefadd'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:59.246962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-30b78d48-03de-4675-80e0-03e28cdefadd admin admin] The token cache is being invalidate because project 1839505f4c284acea50dc0dcc1127a91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:18:59.247258 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-30b78d48-03de-4675-80e0-03e28cdefadd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:59.247917 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-30b78d48-03de-4675-80e0-03e28cdefadd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:59.249402 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1168/2331] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:18:59 2022] DELETE /identity/v3/projects/1839505f4c284acea50dc0dcc1127a91 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:18:59.253480 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4108a1ce-0051-4595-8676-534a05b151b1 service nova] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:59.267261 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4108a1ce-0051-4595-8676-534a05b151b1 service nova] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:59.333575 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2be459-8cfb-467c-b801-8f559349d15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:59.334265 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2be459-8cfb-467c-b801-8f559349d15f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:59.334468 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2be459-8cfb-467c-b801-8f559349d15f admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:59.334721 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2be459-8cfb-467c-b801-8f559349d15f admin admin] PATH_INFO: `/v3/users/36bffe0bbf5f40dcbafd8ee45f0b4af7` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:59.335983 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2be459-8cfb-467c-b801-8f559349d15f admin admin] RBAC: Authorizing `identity:delete_user(user_id=36bffe0bbf5f40dcbafd8ee45f0b4af7)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:59.337882 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2be459-8cfb-467c-b801-8f559349d15f admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:59.396917 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-ae2be459-8cfb-467c-b801-8f559349d15f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36bffe0bbf5f40dcbafd8ee45f0b4af7'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:59.399645 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:18:59.407407 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-ae2be459-8cfb-467c-b801-8f559349d15f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36bffe0bbf5f40dcbafd8ee45f0b4af7'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:59.408079 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-ae2be459-8cfb-467c-b801-8f559349d15f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36bffe0bbf5f40dcbafd8ee45f0b4af7'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:59.408934 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-ae2be459-8cfb-467c-b801-8f559349d15f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36bffe0bbf5f40dcbafd8ee45f0b4af7'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:59.420340 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-ae2be459-8cfb-467c-b801-8f559349d15f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36bffe0bbf5f40dcbafd8ee45f0b4af7'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:59.444208 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1164/2332] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:18:59 2022] DELETE /identity/v3/users/36bffe0bbf5f40dcbafd8ee45f0b4af7 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:18:59.840164 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b78d48-03de-4675-80e0-03e28cdefadd admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:18:59.841650 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-30b78d48-03de-4675-80e0-03e28cdefadd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:18:59.855109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b78d48-03de-4675-80e0-03e28cdefadd admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:18:59.924976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa8b99-2371-473e-92dc-0b8fb6e04557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:18:59.925785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa8b99-2371-473e-92dc-0b8fb6e04557 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:18:59.926162 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa8b99-2371-473e-92dc-0b8fb6e04557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:18:59.926637 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa8b99-2371-473e-92dc-0b8fb6e04557 admin admin] PATH_INFO: `/v3/projects/c254427d1fba47f3849c72342b61a4d5` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:18:59.928004 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fa8b99-2371-473e-92dc-0b8fb6e04557 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c254427d1fba47f3849c72342b61a4d5)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:18:59.931884 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fa8b99-2371-473e-92dc-0b8fb6e04557 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:18:59.986792 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f4fa8b99-2371-473e-92dc-0b8fb6e04557 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c254427d1fba47f3849c72342b61a4d5', 'request_id': 'req-f4fa8b99-2371-473e-92dc-0b8fb6e04557'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:18:59.998354 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f4fa8b99-2371-473e-92dc-0b8fb6e04557 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c254427d1fba47f3849c72342b61a4d5', 'request_id': 'req-f4fa8b99-2371-473e-92dc-0b8fb6e04557'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:00.006455 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f4fa8b99-2371-473e-92dc-0b8fb6e04557 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c254427d1fba47f3849c72342b61a4d5', 'request_id': 'req-f4fa8b99-2371-473e-92dc-0b8fb6e04557'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:00.019524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f4fa8b99-2371-473e-92dc-0b8fb6e04557 admin admin] The token cache is being invalidate because project c254427d1fba47f3849c72342b61a4d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:19:00.019829 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f4fa8b99-2371-473e-92dc-0b8fb6e04557 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:00.020462 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f4fa8b99-2371-473e-92dc-0b8fb6e04557 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:00.022308 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1169/2333] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:18:59 2022] DELETE /identity/v3/projects/c254427d1fba47f3849c72342b61a4d5 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:19:00.027017 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02e51d-2d84-40d5-92d2-1069d0c831b7 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:00.027303 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02e51d-2d84-40d5-92d2-1069d0c831b7 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:00.027576 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02e51d-2d84-40d5-92d2-1069d0c831b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:00.048531 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-ae02e51d-2d84-40d5-92d2-1069d0c831b7 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:19:00.049761 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-ae02e51d-2d84-40d5-92d2-1069d0c831b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:00.139799 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1165/2334] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:19:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:19:00.147800 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968f8b3-fdf7-47a1-bebb-5b254b3f27db None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:00.148652 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968f8b3-fdf7-47a1-bebb-5b254b3f27db None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:00.149170 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968f8b3-fdf7-47a1-bebb-5b254b3f27db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:00.176009 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-0968f8b3-fdf7-47a1-bebb-5b254b3f27db None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:19:00.177351 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-0968f8b3-fdf7-47a1-bebb-5b254b3f27db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:00.186665 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1170/2335] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:19:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:19:00.191156 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02e51d-2d84-40d5-92d2-1069d0c831b7 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:00.209576 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02e51d-2d84-40d5-92d2-1069d0c831b7 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:00.221606 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3686c02f-078e-45c8-93e5-a17bfded15b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:00.222496 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686c02f-078e-45c8-93e5-a17bfded15b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:00.222928 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686c02f-078e-45c8-93e5-a17bfded15b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:00.223299 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686c02f-078e-45c8-93e5-a17bfded15b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:00.224081 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3686c02f-078e-45c8-93e5-a17bfded15b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:19:00.224500 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3686c02f-078e-45c8-93e5-a17bfded15b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:00.226276 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3686c02f-078e-45c8-93e5-a17bfded15b0 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:00.249015 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968f8b3-fdf7-47a1-bebb-5b254b3f27db None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:00.255424 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1166/2336] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:19:00.263393 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968f8b3-fdf7-47a1-bebb-5b254b3f27db None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:00.268126 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7008a-ba7c-45b4-9a47-6532fa8ff11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:00.268400 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7008a-ba7c-45b4-9a47-6532fa8ff11c admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:00.268625 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7008a-ba7c-45b4-9a47-6532fa8ff11c admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:00.268892 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7008a-ba7c-45b4-9a47-6532fa8ff11c admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:00.269945 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d7008a-ba7c-45b4-9a47-6532fa8ff11c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:00.271643 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d7008a-ba7c-45b4-9a47-6532fa8ff11c admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:00.285288 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:19:00.312473 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1171/2337] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:19:00 2022] POST /identity/v3/projects => generated 369 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:19:00.317723 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3686c02f-078e-45c8-93e5-a17bfded15b0 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:00.329417 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3686c02f-078e-45c8-93e5-a17bfded15b0 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:00.409072 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3270937-1c03-4af7-a0de-1406f919b567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:00.410698 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3270937-1c03-4af7-a0de-1406f919b567 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:00.411259 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3270937-1c03-4af7-a0de-1406f919b567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:00.411716 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3270937-1c03-4af7-a0de-1406f919b567 admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:00.412993 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3270937-1c03-4af7-a0de-1406f919b567 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:00.416186 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3270937-1c03-4af7-a0de-1406f919b567 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:00.447028 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1167/2338] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:19:00 2022] POST /identity/v3/users => generated 285 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:19:00.450943 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7008a-ba7c-45b4-9a47-6532fa8ff11c admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:00.463293 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7008a-ba7c-45b4-9a47-6532fa8ff11c admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:00.472278 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55ab3a-d23c-4f52-befd-b7f9b81cae64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:00.472968 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55ab3a-d23c-4f52-befd-b7f9b81cae64 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:00.473835 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55ab3a-d23c-4f52-befd-b7f9b81cae64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:00.474229 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55ab3a-d23c-4f52-befd-b7f9b81cae64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:00.475118 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec55ab3a-d23c-4f52-befd-b7f9b81cae64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:00.476858 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec55ab3a-d23c-4f52-befd-b7f9b81cae64 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:00.494874 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1172/2339] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:19:00.497763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3270937-1c03-4af7-a0de-1406f919b567 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:00.510940 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3270937-1c03-4af7-a0de-1406f919b567 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:00.516435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c5d60-3f04-46e0-a027-748d9c0469e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:00.517197 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c5d60-3f04-46e0-a027-748d9c0469e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:00.517476 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c5d60-3f04-46e0-a027-748d9c0469e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:00.517766 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c5d60-3f04-46e0-a027-748d9c0469e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:00.518418 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c5d60-3f04-46e0-a027-748d9c0469e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:00.522560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c5d60-3f04-46e0-a027-748d9c0469e7 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:00.541337 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1168/2340] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:19:00.545564 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55ab3a-d23c-4f52-befd-b7f9b81cae64 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:00.556442 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55ab3a-d23c-4f52-befd-b7f9b81cae64 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:00.561170 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489d843-d11c-404d-a91f-22e20109e366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:00.562086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489d843-d11c-404d-a91f-22e20109e366 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:00.562368 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489d843-d11c-404d-a91f-22e20109e366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:00.562666 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489d843-d11c-404d-a91f-22e20109e366 admin admin] PATH_INFO: `/v3/projects/5f32761ce8e340949348d4a1d73944cf/users/03d6ed1341464f32bfd9ad9d38e25221/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:00.577450 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489d843-d11c-404d-a91f-22e20109e366 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f32761ce8e340949348d4a1d73944cf, user_id=03d6ed1341464f32bfd9ad9d38e25221, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:00.580200 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489d843-d11c-404d-a91f-22e20109e366 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:00.595860 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1173/2341] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:19:00 2022] PUT /identity/v3/projects/5f32761ce8e340949348d4a1d73944cf/users/03d6ed1341464f32bfd9ad9d38e25221/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:19:00.607406 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5337fc6-d57a-4827-8388-279e50f8c31f None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:00.607774 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5337fc6-d57a-4827-8388-279e50f8c31f None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:00.608046 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5337fc6-d57a-4827-8388-279e50f8c31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:00.638252 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-f5337fc6-d57a-4827-8388-279e50f8c31f None None] MFA Rules not processed for user `03d6ed1341464f32bfd9ad9d38e25221`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:19:00.639653 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-f5337fc6-d57a-4827-8388-279e50f8c31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:00.661586 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:19:00.711172 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1169/2342] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:19:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:19:00.715581 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489d843-d11c-404d-a91f-22e20109e366 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:00.780521 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489d843-d11c-404d-a91f-22e20109e366 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:00.786183 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd4da5-ccd5-42af-9303-878d743b647c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:00.787018 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd4da5-ccd5-42af-9303-878d743b647c admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:00.787364 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd4da5-ccd5-42af-9303-878d743b647c admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:00.787709 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd4da5-ccd5-42af-9303-878d743b647c admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:00.788391 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdd4da5-ccd5-42af-9303-878d743b647c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:00.790462 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdd4da5-ccd5-42af-9303-878d743b647c admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:00.824163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1174/2343] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:19:00 2022] POST /identity/v3/projects => generated 367 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:19:00.825141 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5337fc6-d57a-4827-8388-279e50f8c31f None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:00.837180 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5337fc6-d57a-4827-8388-279e50f8c31f None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:00.909017 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8c274-143e-40c6-81fd-2e2f2b70a4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:00.909633 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8c274-143e-40c6-81fd-2e2f2b70a4df admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:00.909916 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8c274-143e-40c6-81fd-2e2f2b70a4df admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:00.910184 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8c274-143e-40c6-81fd-2e2f2b70a4df admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:00.910903 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf8c274-143e-40c6-81fd-2e2f2b70a4df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:00.912784 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf8c274-143e-40c6-81fd-2e2f2b70a4df admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:00.940902 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1170/2344] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:19:00 2022] POST /identity/v3/users => generated 290 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:19:00.945909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd4da5-ccd5-42af-9303-878d743b647c admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:00.956686 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd4da5-ccd5-42af-9303-878d743b647c admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:00.961753 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96a6c9-9fa3-4886-995e-96b8793d25ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:00.962331 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96a6c9-9fa3-4886-995e-96b8793d25ab admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:00.962588 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96a6c9-9fa3-4886-995e-96b8793d25ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:00.962997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96a6c9-9fa3-4886-995e-96b8793d25ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:00.963549 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e96a6c9-9fa3-4886-995e-96b8793d25ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:00.965379 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e96a6c9-9fa3-4886-995e-96b8793d25ab admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:00.984836 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1175/2345] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:19:00.988977 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8c274-143e-40c6-81fd-2e2f2b70a4df admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:01.000771 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8c274-143e-40c6-81fd-2e2f2b70a4df admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:01.007160 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b8455-1e44-48af-9231-cb9eeb429d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:01.007892 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b8455-1e44-48af-9231-cb9eeb429d4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:01.008184 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b8455-1e44-48af-9231-cb9eeb429d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:01.008503 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b8455-1e44-48af-9231-cb9eeb429d4e admin admin] PATH_INFO: `/v3/projects/71c14d53bb6442b0a531931adff683b8/users/1f1d835cc7d64e69865ca18ef47a60fa/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:01.027219 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b8455-1e44-48af-9231-cb9eeb429d4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=71c14d53bb6442b0a531931adff683b8, user_id=1f1d835cc7d64e69865ca18ef47a60fa, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:01.030176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b8455-1e44-48af-9231-cb9eeb429d4e admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:01.046780 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1171/2346] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:19:00 2022] PUT /identity/v3/projects/71c14d53bb6442b0a531931adff683b8/users/1f1d835cc7d64e69865ca18ef47a60fa/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:19:01.052197 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96a6c9-9fa3-4886-995e-96b8793d25ab admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:01.066268 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96a6c9-9fa3-4886-995e-96b8793d25ab admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:01.129858 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7914c3fd-11e5-4f09-8a6b-35a658f4d261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:01.130728 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914c3fd-11e5-4f09-8a6b-35a658f4d261 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:01.131161 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914c3fd-11e5-4f09-8a6b-35a658f4d261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:01.131504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914c3fd-11e5-4f09-8a6b-35a658f4d261 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:01.132241 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7914c3fd-11e5-4f09-8a6b-35a658f4d261 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:01.134528 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7914c3fd-11e5-4f09-8a6b-35a658f4d261 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:01.149487 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1176/2347] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:19:01.153610 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b8455-1e44-48af-9231-cb9eeb429d4e admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:01.169902 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b8455-1e44-48af-9231-cb9eeb429d4e admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:01.176034 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daefbad7-76d0-48a5-96fd-f9b575170ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:01.176947 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daefbad7-76d0-48a5-96fd-f9b575170ba5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:01.177194 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daefbad7-76d0-48a5-96fd-f9b575170ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:01.177500 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daefbad7-76d0-48a5-96fd-f9b575170ba5 admin admin] PATH_INFO: `/v3/projects/71c14d53bb6442b0a531931adff683b8/users/1f1d835cc7d64e69865ca18ef47a60fa/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:01.179900 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daefbad7-76d0-48a5-96fd-f9b575170ba5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71c14d53bb6442b0a531931adff683b8, user_id=1f1d835cc7d64e69865ca18ef47a60fa, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:01.197173 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daefbad7-76d0-48a5-96fd-f9b575170ba5 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:01.219786 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1172/2348] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:19:01 2022] PUT /identity/v3/projects/71c14d53bb6442b0a531931adff683b8/users/1f1d835cc7d64e69865ca18ef47a60fa/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:19:01.231521 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7bf12-325c-461c-aba2-905a9d450365 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:01.231845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7bf12-325c-461c-aba2-905a9d450365 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:01.232028 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7bf12-325c-461c-aba2-905a9d450365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:01.278034 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-88a7bf12-325c-461c-aba2-905a9d450365 None None] MFA Rules not processed for user `1f1d835cc7d64e69865ca18ef47a60fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:19:01.279437 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-88a7bf12-325c-461c-aba2-905a9d450365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:01.301090 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:19:01.372565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1177/2349] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:19:01 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:19:01.380385 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daefbad7-76d0-48a5-96fd-f9b575170ba5 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:01.382202 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-daefbad7-76d0-48a5-96fd-f9b575170ba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:01.504656 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daefbad7-76d0-48a5-96fd-f9b575170ba5 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:01.509503 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cafc68-da35-440e-97ee-0c3ad5b9f2f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:01.510264 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cafc68-da35-440e-97ee-0c3ad5b9f2f6 service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:01.510628 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cafc68-da35-440e-97ee-0c3ad5b9f2f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:01.511175 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cafc68-da35-440e-97ee-0c3ad5b9f2f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:01.521818 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cafc68-da35-440e-97ee-0c3ad5b9f2f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:01.523285 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cafc68-da35-440e-97ee-0c3ad5b9f2f6 service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:01.534410 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1173/2350] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:19:01 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:19:01.604188 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a7bf12-325c-461c-aba2-905a9d450365 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:01.605753 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-88a7bf12-325c-461c-aba2-905a9d450365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:01.623233 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a7bf12-325c-461c-aba2-905a9d450365 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:01.628385 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47cbba1-1769-4796-8599-63e01af306ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:01.629025 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47cbba1-1769-4796-8599-63e01af306ac service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:01.629305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47cbba1-1769-4796-8599-63e01af306ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:01.629579 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47cbba1-1769-4796-8599-63e01af306ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:01.710222 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47cbba1-1769-4796-8599-63e01af306ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:01.711716 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47cbba1-1769-4796-8599-63e01af306ac service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:01.722464 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1178/2351] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:19:01 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:19:02.256174 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a30e6-f0ef-473e-883c-0de4bd9bfbea None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:02.256588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a30e6-f0ef-473e-883c-0de4bd9bfbea None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:02.257139 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a30e6-f0ef-473e-883c-0de4bd9bfbea None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:02.258873 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1174/2352] 10.222.0.42 () {54 vars in 921 bytes} [Mon Sep 12 14:19:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:19:02.269646 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a948f4-0193-4c51-886b-6ed8a5721c12 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:02.270162 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a948f4-0193-4c51-886b-6ed8a5721c12 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:02.270162 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a948f4-0193-4c51-886b-6ed8a5721c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:02.291045 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-49a948f4-0193-4c51-886b-6ed8a5721c12 None None] MFA Rules not processed for user `ce508bb9aa164a7fb1831a9b5ff5b932`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:19:02.292262 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-49a948f4-0193-4c51-886b-6ed8a5721c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:02.365185 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1179/2353] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Sep 12 14:19:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:19:02.376347 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a30e6-f0ef-473e-883c-0de4bd9bfbea None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:02.378195 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-822a30e6-f0ef-473e-883c-0de4bd9bfbea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:02.395227 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a30e6-f0ef-473e-883c-0de4bd9bfbea None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:02.400296 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74416c78-8828-4c87-9181-9e4883a96fa6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce508bb9aa164a7fb1831a9b5ff5b932', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:02.400893 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74416c78-8828-4c87-9181-9e4883a96fa6 service placement] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:02.401170 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74416c78-8828-4c87-9181-9e4883a96fa6 service placement] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:02.401441 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74416c78-8828-4c87-9181-9e4883a96fa6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:02.410921 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74416c78-8828-4c87-9181-9e4883a96fa6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:02.412505 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74416c78-8828-4c87-9181-9e4883a96fa6 service placement] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:02.423201 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1175/2354] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:19:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:19:05.263960 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a948f4-0193-4c51-886b-6ed8a5721c12 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:05.355004 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:19:05.380357 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a948f4-0193-4c51-886b-6ed8a5721c12 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:05.385644 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744b123-f64a-43ca-ac46-944eeed834b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:05.386914 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744b123-f64a-43ca-ac46-944eeed834b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:05.387423 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744b123-f64a-43ca-ac46-944eeed834b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:05.388041 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744b123-f64a-43ca-ac46-944eeed834b3 admin admin] PATH_INFO: `/v3/users/03d6ed1341464f32bfd9ad9d38e25221` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:05.389430 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1744b123-f64a-43ca-ac46-944eeed834b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03d6ed1341464f32bfd9ad9d38e25221)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:05.391643 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1744b123-f64a-43ca-ac46-944eeed834b3 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:05.461656 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-1744b123-f64a-43ca-ac46-944eeed834b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03d6ed1341464f32bfd9ad9d38e25221'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:05.470420 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-1744b123-f64a-43ca-ac46-944eeed834b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03d6ed1341464f32bfd9ad9d38e25221'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:05.471134 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-1744b123-f64a-43ca-ac46-944eeed834b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03d6ed1341464f32bfd9ad9d38e25221'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:05.471852 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-1744b123-f64a-43ca-ac46-944eeed834b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03d6ed1341464f32bfd9ad9d38e25221'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:05.483971 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-1744b123-f64a-43ca-ac46-944eeed834b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03d6ed1341464f32bfd9ad9d38e25221'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:05.504561 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1180/2355] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:19:05 2022] DELETE /identity/v3/users/03d6ed1341464f32bfd9ad9d38e25221 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:19:05.508794 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76111bf5-8bc7-421c-bf0e-e3ca07b19f0f None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:05.509283 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76111bf5-8bc7-421c-bf0e-e3ca07b19f0f None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:05.509414 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76111bf5-8bc7-421c-bf0e-e3ca07b19f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:05.517950 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:19:05.529992 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-76111bf5-8bc7-421c-bf0e-e3ca07b19f0f None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:19:05.531679 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-76111bf5-8bc7-421c-bf0e-e3ca07b19f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:05.601001 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1176/2356] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:19:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:19:05.611337 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744b123-f64a-43ca-ac46-944eeed834b3 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:05.612860 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1744b123-f64a-43ca-ac46-944eeed834b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:05.681128 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744b123-f64a-43ca-ac46-944eeed834b3 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:05.685088 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5a8f4-d71d-4f0f-8e07-dfe9a3fce24d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:05.685795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5a8f4-d71d-4f0f-8e07-dfe9a3fce24d service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:05.686876 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5a8f4-d71d-4f0f-8e07-dfe9a3fce24d service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:05.686876 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5a8f4-d71d-4f0f-8e07-dfe9a3fce24d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:05.696538 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f5a8f4-d71d-4f0f-8e07-dfe9a3fce24d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:05.698174 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f5a8f4-d71d-4f0f-8e07-dfe9a3fce24d service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:05.708659 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1181/2357] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:19:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:19:06.218511 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76111bf5-8bc7-421c-bf0e-e3ca07b19f0f None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:06.221640 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-76111bf5-8bc7-421c-bf0e-e3ca07b19f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:06.255146 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76111bf5-8bc7-421c-bf0e-e3ca07b19f0f None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:06.264227 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a5a64-0f7f-40d2-b238-690c2b6bbbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:06.264913 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a5a64-0f7f-40d2-b238-690c2b6bbbbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:06.265197 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a5a64-0f7f-40d2-b238-690c2b6bbbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:06.265980 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a5a64-0f7f-40d2-b238-690c2b6bbbbd admin admin] PATH_INFO: `/v3/projects/5f32761ce8e340949348d4a1d73944cf` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:06.267228 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808a5a64-0f7f-40d2-b238-690c2b6bbbbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f32761ce8e340949348d4a1d73944cf)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:06.269326 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808a5a64-0f7f-40d2-b238-690c2b6bbbbd admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:06.328659 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-808a5a64-0f7f-40d2-b238-690c2b6bbbbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f32761ce8e340949348d4a1d73944cf', 'request_id': 'req-808a5a64-0f7f-40d2-b238-690c2b6bbbbd'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:06.334635 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-808a5a64-0f7f-40d2-b238-690c2b6bbbbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f32761ce8e340949348d4a1d73944cf', 'request_id': 'req-808a5a64-0f7f-40d2-b238-690c2b6bbbbd'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:06.343207 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-808a5a64-0f7f-40d2-b238-690c2b6bbbbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f32761ce8e340949348d4a1d73944cf', 'request_id': 'req-808a5a64-0f7f-40d2-b238-690c2b6bbbbd'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:06.354620 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-808a5a64-0f7f-40d2-b238-690c2b6bbbbd admin admin] The token cache is being invalidate because project 5f32761ce8e340949348d4a1d73944cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:19:06.354945 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-808a5a64-0f7f-40d2-b238-690c2b6bbbbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:06.355588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-808a5a64-0f7f-40d2-b238-690c2b6bbbbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:06.358968 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1177/2358] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:19:06 2022] DELETE /identity/v3/projects/5f32761ce8e340949348d4a1d73944cf => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:19:06.361776 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5a8f4-d71d-4f0f-8e07-dfe9a3fce24d service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:06.371710 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5a8f4-d71d-4f0f-8e07-dfe9a3fce24d service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:06.431504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f831db1-81c7-4936-8298-336d84cbb410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:06.432133 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f831db1-81c7-4936-8298-336d84cbb410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:06.432405 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f831db1-81c7-4936-8298-336d84cbb410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:06.432675 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f831db1-81c7-4936-8298-336d84cbb410 admin admin] PATH_INFO: `/v3/users/1f1d835cc7d64e69865ca18ef47a60fa` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:06.433787 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f831db1-81c7-4936-8298-336d84cbb410 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f1d835cc7d64e69865ca18ef47a60fa)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:06.435721 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f831db1-81c7-4936-8298-336d84cbb410 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:06.446563 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:19:06.498938 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-8f831db1-81c7-4936-8298-336d84cbb410 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f1d835cc7d64e69865ca18ef47a60fa'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:06.507363 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-8f831db1-81c7-4936-8298-336d84cbb410 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f1d835cc7d64e69865ca18ef47a60fa'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:06.508104 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-8f831db1-81c7-4936-8298-336d84cbb410 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f1d835cc7d64e69865ca18ef47a60fa'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:06.508718 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-8f831db1-81c7-4936-8298-336d84cbb410 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f1d835cc7d64e69865ca18ef47a60fa'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:06.517085 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-8f831db1-81c7-4936-8298-336d84cbb410 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f1d835cc7d64e69865ca18ef47a60fa'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:06.534228 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1182/2359] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:19:06 2022] DELETE /identity/v3/users/1f1d835cc7d64e69865ca18ef47a60fa => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:19:06.932234 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a5a64-0f7f-40d2-b238-690c2b6bbbbd admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:06.934270 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-808a5a64-0f7f-40d2-b238-690c2b6bbbbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:06.957616 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a5a64-0f7f-40d2-b238-690c2b6bbbbd admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:07.037324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac5d972-983f-4efe-9e55-32f931c1fc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:07.038200 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5d972-983f-4efe-9e55-32f931c1fc16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:07.038544 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5d972-983f-4efe-9e55-32f931c1fc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:07.038944 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5d972-983f-4efe-9e55-32f931c1fc16 admin admin] PATH_INFO: `/v3/projects/71c14d53bb6442b0a531931adff683b8` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:07.040324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac5d972-983f-4efe-9e55-32f931c1fc16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71c14d53bb6442b0a531931adff683b8)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:07.042276 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac5d972-983f-4efe-9e55-32f931c1fc16 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:07.085760 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:19:07.103711 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-2ac5d972-983f-4efe-9e55-32f931c1fc16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71c14d53bb6442b0a531931adff683b8', 'request_id': 'req-2ac5d972-983f-4efe-9e55-32f931c1fc16'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:07.112119 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-2ac5d972-983f-4efe-9e55-32f931c1fc16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71c14d53bb6442b0a531931adff683b8', 'request_id': 'req-2ac5d972-983f-4efe-9e55-32f931c1fc16'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:07.125928 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-2ac5d972-983f-4efe-9e55-32f931c1fc16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71c14d53bb6442b0a531931adff683b8', 'request_id': 'req-2ac5d972-983f-4efe-9e55-32f931c1fc16'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:07.145481 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-2ac5d972-983f-4efe-9e55-32f931c1fc16 admin admin] The token cache is being invalidate because project 71c14d53bb6442b0a531931adff683b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:19:07.146043 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-2ac5d972-983f-4efe-9e55-32f931c1fc16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:07.146828 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-2ac5d972-983f-4efe-9e55-32f931c1fc16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:07.148744 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1178/2360] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:19:06 2022] DELETE /identity/v3/projects/71c14d53bb6442b0a531931adff683b8 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:19:07.153922 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78f8c7-2502-4890-9e25-338e563d5f38 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:07.154199 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78f8c7-2502-4890-9e25-338e563d5f38 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:07.154474 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78f8c7-2502-4890-9e25-338e563d5f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:07.202165 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-0f78f8c7-2502-4890-9e25-338e563d5f38 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:19:07.204842 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-0f78f8c7-2502-4890-9e25-338e563d5f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:07.282870 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1183/2361] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:19:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:19:07.287715 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac5d972-983f-4efe-9e55-32f931c1fc16 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:07.289130 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-2ac5d972-983f-4efe-9e55-32f931c1fc16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:07.290408 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15332921-7d53-4102-ad43-16333de779a9 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:07.290747 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15332921-7d53-4102-ad43-16333de779a9 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:07.291058 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15332921-7d53-4102-ad43-16333de779a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:07.303945 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac5d972-983f-4efe-9e55-32f931c1fc16 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:07.310975 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed5780-5905-4b7b-b63a-c44b20a50b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:07.311150 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-15332921-7d53-4102-ad43-16333de779a9 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:19:07.312004 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed5780-5905-4b7b-b63a-c44b20a50b2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:07.312492 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed5780-5905-4b7b-b63a-c44b20a50b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:07.312628 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-15332921-7d53-4102-ad43-16333de779a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:07.312896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed5780-5905-4b7b-b63a-c44b20a50b2c admin admin] PATH_INFO: `/v3/users/67c00c7b37ba44aa9ca0e4555d32ec90` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:07.314166 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ed5780-5905-4b7b-b63a-c44b20a50b2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=67c00c7b37ba44aa9ca0e4555d32ec90)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:07.316141 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ed5780-5905-4b7b-b63a-c44b20a50b2c admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:07.320438 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1184/2362] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:19:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:19:07.324541 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15332921-7d53-4102-ad43-16333de779a9 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:07.333599 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15332921-7d53-4102-ad43-16333de779a9 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:07.338689 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ddb66-d9f2-476b-a596-9278ba07213d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:07.340109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ddb66-d9f2-476b-a596-9278ba07213d admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:07.340907 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ddb66-d9f2-476b-a596-9278ba07213d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:07.341195 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ddb66-d9f2-476b-a596-9278ba07213d admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:07.341849 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3ddb66-d9f2-476b-a596-9278ba07213d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:19:07.342409 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3ddb66-d9f2-476b-a596-9278ba07213d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:07.345032 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3ddb66-d9f2-476b-a596-9278ba07213d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:07.358158 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1185/2363] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:19:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:19:07.363087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ddb66-d9f2-476b-a596-9278ba07213d admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:07.369863 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-b5ed5780-5905-4b7b-b63a-c44b20a50b2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67c00c7b37ba44aa9ca0e4555d32ec90'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:07.373959 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ddb66-d9f2-476b-a596-9278ba07213d admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:07.377648 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-b5ed5780-5905-4b7b-b63a-c44b20a50b2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67c00c7b37ba44aa9ca0e4555d32ec90'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:07.378318 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-b5ed5780-5905-4b7b-b63a-c44b20a50b2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67c00c7b37ba44aa9ca0e4555d32ec90'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:07.378959 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-b5ed5780-5905-4b7b-b63a-c44b20a50b2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67c00c7b37ba44aa9ca0e4555d32ec90'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:07.379429 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bdd617-5985-4bcc-83fd-ec9b6a4b3943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:07.380029 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bdd617-5985-4bcc-83fd-ec9b6a4b3943 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:07.380311 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bdd617-5985-4bcc-83fd-ec9b6a4b3943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:07.380586 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bdd617-5985-4bcc-83fd-ec9b6a4b3943 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:07.381221 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bdd617-5985-4bcc-83fd-ec9b6a4b3943 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:07.383133 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bdd617-5985-4bcc-83fd-ec9b6a4b3943 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:07.386515 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-b5ed5780-5905-4b7b-b63a-c44b20a50b2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67c00c7b37ba44aa9ca0e4555d32ec90'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:07.404930 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1179/2364] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:19:07 2022] DELETE /identity/v3/users/67c00c7b37ba44aa9ca0e4555d32ec90 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:19:07.412469 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1186/2365] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:19:07 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:19:07.416829 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed5780-5905-4b7b-b63a-c44b20a50b2c admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:07.418252 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-b5ed5780-5905-4b7b-b63a-c44b20a50b2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:07.489684 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed5780-5905-4b7b-b63a-c44b20a50b2c admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:07.526406 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:19:07.551479 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c73a2dd-fc2f-4638-87f8-f396e2af64dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:07.552089 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73a2dd-fc2f-4638-87f8-f396e2af64dd admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:07.552362 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73a2dd-fc2f-4638-87f8-f396e2af64dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:07.552630 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73a2dd-fc2f-4638-87f8-f396e2af64dd admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:07.553253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c73a2dd-fc2f-4638-87f8-f396e2af64dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:07.555216 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c73a2dd-fc2f-4638-87f8-f396e2af64dd admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:07.577359 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1180/2366] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:19:07 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:19:07.582204 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bdd617-5985-4bcc-83fd-ec9b6a4b3943 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:07.590788 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c73a2dd-fc2f-4638-87f8-f396e2af64dd admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:07.592644 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-6c73a2dd-fc2f-4638-87f8-f396e2af64dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:07.595591 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bdd617-5985-4bcc-83fd-ec9b6a4b3943 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:07.600482 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0fc56b-4eba-418f-80bd-6e1213994f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:07.601088 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fc56b-4eba-418f-80bd-6e1213994f21 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:07.601358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fc56b-4eba-418f-80bd-6e1213994f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:07.601648 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fc56b-4eba-418f-80bd-6e1213994f21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:07.602265 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0fc56b-4eba-418f-80bd-6e1213994f21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:07.603876 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0fc56b-4eba-418f-80bd-6e1213994f21 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:07.606356 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c73a2dd-fc2f-4638-87f8-f396e2af64dd admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:07.612144 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db744d0-8503-4897-b95d-5fbf4a9cbcfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:07.612735 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db744d0-8503-4897-b95d-5fbf4a9cbcfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:07.613009 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db744d0-8503-4897-b95d-5fbf4a9cbcfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:07.613275 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db744d0-8503-4897-b95d-5fbf4a9cbcfa admin admin] PATH_INFO: `/v3/projects/40918a5eb3134a00999618cb65dedbbd` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:07.614742 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1187/2367] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:19:07.614905 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db744d0-8503-4897-b95d-5fbf4a9cbcfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=40918a5eb3134a00999618cb65dedbbd)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:07.616858 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db744d0-8503-4897-b95d-5fbf4a9cbcfa admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:07.619182 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0fc56b-4eba-418f-80bd-6e1213994f21 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:07.629390 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0fc56b-4eba-418f-80bd-6e1213994f21 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:07.635290 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13bedb9-5ab7-41bf-a6dc-6dc9590cd68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:07.635885 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13bedb9-5ab7-41bf-a6dc-6dc9590cd68b admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:07.636171 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13bedb9-5ab7-41bf-a6dc-6dc9590cd68b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:07.636445 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13bedb9-5ab7-41bf-a6dc-6dc9590cd68b admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:07.637087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13bedb9-5ab7-41bf-a6dc-6dc9590cd68b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:07.638784 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13bedb9-5ab7-41bf-a6dc-6dc9590cd68b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:07.649825 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1188/2368] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:19:07.653871 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13bedb9-5ab7-41bf-a6dc-6dc9590cd68b admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:07.664763 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13bedb9-5ab7-41bf-a6dc-6dc9590cd68b admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:07.667061 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-4db744d0-8503-4897-b95d-5fbf4a9cbcfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40918a5eb3134a00999618cb65dedbbd', 'request_id': 'req-4db744d0-8503-4897-b95d-5fbf4a9cbcfa'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:07.673006 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-4db744d0-8503-4897-b95d-5fbf4a9cbcfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40918a5eb3134a00999618cb65dedbbd', 'request_id': 'req-4db744d0-8503-4897-b95d-5fbf4a9cbcfa'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:07.681158 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-4db744d0-8503-4897-b95d-5fbf4a9cbcfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40918a5eb3134a00999618cb65dedbbd', 'request_id': 'req-4db744d0-8503-4897-b95d-5fbf4a9cbcfa'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:07.688736 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:19:07.695898 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-4db744d0-8503-4897-b95d-5fbf4a9cbcfa admin admin] The token cache is being invalidate because project 40918a5eb3134a00999618cb65dedbbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:19:07.696279 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-4db744d0-8503-4897-b95d-5fbf4a9cbcfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:07.696975 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-4db744d0-8503-4897-b95d-5fbf4a9cbcfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:07.698456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1181/2369] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:19:07 2022] DELETE /identity/v3/projects/40918a5eb3134a00999618cb65dedbbd => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:19:07.702462 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db744d0-8503-4897-b95d-5fbf4a9cbcfa admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:07.704447 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-4db744d0-8503-4897-b95d-5fbf4a9cbcfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:07.729433 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193005b-a3b1-45f5-8e32-8df5dbdeb3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:07.730095 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193005b-a3b1-45f5-8e32-8df5dbdeb3f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:07.730432 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193005b-a3b1-45f5-8e32-8df5dbdeb3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:07.730714 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193005b-a3b1-45f5-8e32-8df5dbdeb3f1 admin admin] PATH_INFO: `/v3/projects/2bc1c645366f4ecc82d65726ffac1c85/users/1fa5bf227bbc49f29fdc247d79f2583a/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:07.745338 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193005b-a3b1-45f5-8e32-8df5dbdeb3f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bc1c645366f4ecc82d65726ffac1c85, user_id=1fa5bf227bbc49f29fdc247d79f2583a, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:07.748030 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193005b-a3b1-45f5-8e32-8df5dbdeb3f1 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:07.770923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1189/2370] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:19:07 2022] PUT /identity/v3/projects/2bc1c645366f4ecc82d65726ffac1c85/users/1fa5bf227bbc49f29fdc247d79f2583a/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:19:07.771801 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db744d0-8503-4897-b95d-5fbf4a9cbcfa admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:07.775431 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04da94-e083-4286-8207-1b221bec7f12 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:07.775761 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04da94-e083-4286-8207-1b221bec7f12 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:07.776043 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04da94-e083-4286-8207-1b221bec7f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:07.810714 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-bd04da94-e083-4286-8207-1b221bec7f12 None None] MFA Rules not processed for user `1fa5bf227bbc49f29fdc247d79f2583a`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:19:07.813042 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-bd04da94-e083-4286-8207-1b221bec7f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:07.839999 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:19:07.847463 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95732d78-2cc0-4540-98b1-18e406d31071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:07.848189 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95732d78-2cc0-4540-98b1-18e406d31071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:07.848636 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95732d78-2cc0-4540-98b1-18e406d31071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:07.849148 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95732d78-2cc0-4540-98b1-18e406d31071 admin admin] PATH_INFO: `/v3/users/db6bb10286024e5f9cc3c880469e2b16` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:07.850730 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95732d78-2cc0-4540-98b1-18e406d31071 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db6bb10286024e5f9cc3c880469e2b16)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:07.853222 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95732d78-2cc0-4540-98b1-18e406d31071 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:07.889313 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1190/2371] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:19:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:19:07.894707 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd04da94-e083-4286-8207-1b221bec7f12 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:07.896075 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-bd04da94-e083-4286-8207-1b221bec7f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:07.906431 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-95732d78-2cc0-4540-98b1-18e406d31071 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db6bb10286024e5f9cc3c880469e2b16'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:07.912955 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd04da94-e083-4286-8207-1b221bec7f12 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:07.915895 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-95732d78-2cc0-4540-98b1-18e406d31071 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db6bb10286024e5f9cc3c880469e2b16'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:07.916828 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-95732d78-2cc0-4540-98b1-18e406d31071 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db6bb10286024e5f9cc3c880469e2b16'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:07.917521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-95732d78-2cc0-4540-98b1-18e406d31071 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db6bb10286024e5f9cc3c880469e2b16'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:07.918852 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01de33d-2c13-45e9-8930-5a1ebea41dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:07.919072 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01de33d-2c13-45e9-8930-5a1ebea41dba admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:07.919345 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01de33d-2c13-45e9-8930-5a1ebea41dba admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:07.919616 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01de33d-2c13-45e9-8930-5a1ebea41dba admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:07.920264 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01de33d-2c13-45e9-8930-5a1ebea41dba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:07.922207 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01de33d-2c13-45e9-8930-5a1ebea41dba admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:07.925293 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-95732d78-2cc0-4540-98b1-18e406d31071 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db6bb10286024e5f9cc3c880469e2b16'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:07.946919 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1182/2372] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:19:07 2022] DELETE /identity/v3/users/db6bb10286024e5f9cc3c880469e2b16 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:19:07.961412 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1191/2373] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:19:07 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:19:07.962387 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95732d78-2cc0-4540-98b1-18e406d31071 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:07.963877 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-95732d78-2cc0-4540-98b1-18e406d31071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:08.030272 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95732d78-2cc0-4540-98b1-18e406d31071 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:08.087510 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e134390-9b14-4ef6-904a-1d653d8cbba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:08.088052 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e134390-9b14-4ef6-904a-1d653d8cbba0 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:08.088333 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e134390-9b14-4ef6-904a-1d653d8cbba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:08.088607 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e134390-9b14-4ef6-904a-1d653d8cbba0 admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:08.089358 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e134390-9b14-4ef6-904a-1d653d8cbba0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:08.091349 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e134390-9b14-4ef6-904a-1d653d8cbba0 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:08.114259 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1183/2374] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:19:07 2022] POST /identity/v3/users => generated 299 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:19:08.118531 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01de33d-2c13-45e9-8930-5a1ebea41dba admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:08.125702 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e134390-9b14-4ef6-904a-1d653d8cbba0 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:08.128428 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-6e134390-9b14-4ef6-904a-1d653d8cbba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:08.130277 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01de33d-2c13-45e9-8930-5a1ebea41dba admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:08.136067 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31bf1ee-a970-4cdb-b2da-392f01367d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:08.136882 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31bf1ee-a970-4cdb-b2da-392f01367d89 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:08.137411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31bf1ee-a970-4cdb-b2da-392f01367d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:08.137871 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31bf1ee-a970-4cdb-b2da-392f01367d89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:08.138690 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31bf1ee-a970-4cdb-b2da-392f01367d89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:08.141035 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31bf1ee-a970-4cdb-b2da-392f01367d89 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:08.143219 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e134390-9b14-4ef6-904a-1d653d8cbba0 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:08.149115 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6808dc5-4b48-43be-913f-31413203d78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:08.149115 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6808dc5-4b48-43be-913f-31413203d78f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:08.149386 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6808dc5-4b48-43be-913f-31413203d78f admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:08.149586 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6808dc5-4b48-43be-913f-31413203d78f admin admin] PATH_INFO: `/v3/projects/8f268344ee214486819057a236e65377` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:08.150872 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6808dc5-4b48-43be-913f-31413203d78f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f268344ee214486819057a236e65377)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:08.152699 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6808dc5-4b48-43be-913f-31413203d78f admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:08.156767 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1192/2375] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:19:08.162472 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31bf1ee-a970-4cdb-b2da-392f01367d89 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:08.165907 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:19:08.177133 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31bf1ee-a970-4cdb-b2da-392f01367d89 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:08.182256 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92ef42-02b1-480c-b82a-798d971c03c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:08.182951 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92ef42-02b1-480c-b82a-798d971c03c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:08.183229 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92ef42-02b1-480c-b82a-798d971c03c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:08.183510 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92ef42-02b1-480c-b82a-798d971c03c1 admin admin] PATH_INFO: `/v3/projects/a25eca2687954bd795e628b1d373d309/users/05c76029555447baae6fa7a641e77c01/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:08.200236 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf92ef42-02b1-480c-b82a-798d971c03c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a25eca2687954bd795e628b1d373d309, user_id=05c76029555447baae6fa7a641e77c01, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:08.202957 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf92ef42-02b1-480c-b82a-798d971c03c1 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:08.212704 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-b6808dc5-4b48-43be-913f-31413203d78f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f268344ee214486819057a236e65377', 'request_id': 'req-b6808dc5-4b48-43be-913f-31413203d78f'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:08.216010 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1193/2376] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:19:08 2022] PUT /identity/v3/projects/a25eca2687954bd795e628b1d373d309/users/05c76029555447baae6fa7a641e77c01/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:19:08.218742 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-b6808dc5-4b48-43be-913f-31413203d78f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f268344ee214486819057a236e65377', 'request_id': 'req-b6808dc5-4b48-43be-913f-31413203d78f'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:08.219861 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92ef42-02b1-480c-b82a-798d971c03c1 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:08.226685 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-b6808dc5-4b48-43be-913f-31413203d78f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f268344ee214486819057a236e65377', 'request_id': 'req-b6808dc5-4b48-43be-913f-31413203d78f'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:08.242656 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-b6808dc5-4b48-43be-913f-31413203d78f admin admin] The token cache is being invalidate because project 8f268344ee214486819057a236e65377 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:19:08.243116 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-b6808dc5-4b48-43be-913f-31413203d78f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:08.243843 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-b6808dc5-4b48-43be-913f-31413203d78f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:08.245402 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1184/2377] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:19:08 2022] DELETE /identity/v3/projects/8f268344ee214486819057a236e65377 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:19:08.250720 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ec845-29b7-4ae6-81f3-efba54a1365f None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:08.251024 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ec845-29b7-4ae6-81f3-efba54a1365f None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:08.251298 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ec845-29b7-4ae6-81f3-efba54a1365f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:08.265191 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:19:08.270397 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-f07ec845-29b7-4ae6-81f3-efba54a1365f None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:19:08.271599 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-f07ec845-29b7-4ae6-81f3-efba54a1365f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:08.285949 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92ef42-02b1-480c-b82a-798d971c03c1 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:08.290643 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1fc742-ff88-496b-a934-52df909c8a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:08.291317 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1fc742-ff88-496b-a934-52df909c8a09 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:08.291601 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1fc742-ff88-496b-a934-52df909c8a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:08.291876 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1fc742-ff88-496b-a934-52df909c8a09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:08.292516 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1fc742-ff88-496b-a934-52df909c8a09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:08.294083 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1fc742-ff88-496b-a934-52df909c8a09 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:08.309483 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1194/2378] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:19:08.310353 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1fc742-ff88-496b-a934-52df909c8a09 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:08.313678 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-2e1fc742-ff88-496b-a934-52df909c8a09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:08.326605 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1fc742-ff88-496b-a934-52df909c8a09 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:08.331362 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09fa13-1d88-4f15-856d-0e8aef6e42d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:08.332960 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09fa13-1d88-4f15-856d-0e8aef6e42d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:08.332960 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09fa13-1d88-4f15-856d-0e8aef6e42d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:08.332960 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09fa13-1d88-4f15-856d-0e8aef6e42d7 admin admin] PATH_INFO: `/v3/projects/a25eca2687954bd795e628b1d373d309/users/05c76029555447baae6fa7a641e77c01/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:08.334638 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be09fa13-1d88-4f15-856d-0e8aef6e42d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a25eca2687954bd795e628b1d373d309, user_id=05c76029555447baae6fa7a641e77c01, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:08.337264 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be09fa13-1d88-4f15-856d-0e8aef6e42d7 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:08.346705 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1185/2379] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:19:08.348102 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1195/2380] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:19:08 2022] PUT /identity/v3/projects/a25eca2687954bd795e628b1d373d309/users/05c76029555447baae6fa7a641e77c01/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:19:08.354558 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb51d87-e254-4664-bc2a-793f5829c84a None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:08.354860 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb51d87-e254-4664-bc2a-793f5829c84a None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:08.355149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb51d87-e254-4664-bc2a-793f5829c84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:08.358004 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d4b40-35e2-4081-8301-5ad5e0788c1a None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:08.358288 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d4b40-35e2-4081-8301-5ad5e0788c1a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:08.358565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d4b40-35e2-4081-8301-5ad5e0788c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:08.376239 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-0fb51d87-e254-4664-bc2a-793f5829c84a None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:19:08.377446 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-0fb51d87-e254-4664-bc2a-793f5829c84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:08.387739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-916d4b40-35e2-4081-8301-5ad5e0788c1a None None] MFA Rules not processed for user `05c76029555447baae6fa7a641e77c01`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:19:08.388903 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-916d4b40-35e2-4081-8301-5ad5e0788c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:08.441660 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1186/2381] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:19:08.445806 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb51d87-e254-4664-bc2a-793f5829c84a None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:08.454456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb51d87-e254-4664-bc2a-793f5829c84a None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:08.459147 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1196/2382] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:19:08 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:19:08.459353 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904aa016-e76c-4bb9-a888-6577bb3bb0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:08.460096 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904aa016-e76c-4bb9-a888-6577bb3bb0df admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:08.460464 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904aa016-e76c-4bb9-a888-6577bb3bb0df admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:08.460802 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904aa016-e76c-4bb9-a888-6577bb3bb0df admin admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:08.461512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904aa016-e76c-4bb9-a888-6577bb3bb0df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:19:08.461883 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904aa016-e76c-4bb9-a888-6577bb3bb0df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:08.463777 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904aa016-e76c-4bb9-a888-6577bb3bb0df admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:08.469693 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d4b40-35e2-4081-8301-5ad5e0788c1a None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:08.471118 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-916d4b40-35e2-4081-8301-5ad5e0788c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:08.476446 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1187/2383] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:19:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:19:08.481092 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904aa016-e76c-4bb9-a888-6577bb3bb0df admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:08.492351 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904aa016-e76c-4bb9-a888-6577bb3bb0df admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:08.503282 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ddd862-11aa-46f6-8f20-88ea4bcdb6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:08.503882 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddd862-11aa-46f6-8f20-88ea4bcdb6a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:08.504229 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddd862-11aa-46f6-8f20-88ea4bcdb6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:08.504505 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddd862-11aa-46f6-8f20-88ea4bcdb6a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:08.505130 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ddd862-11aa-46f6-8f20-88ea4bcdb6a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:08.507151 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ddd862-11aa-46f6-8f20-88ea4bcdb6a0 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:08.539082 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1188/2384] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:19:08 2022] POST /identity/v3/projects => generated 371 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:19:08.542781 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ddd862-11aa-46f6-8f20-88ea4bcdb6a0 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:08.544503 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d4b40-35e2-4081-8301-5ad5e0788c1a None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:08.547659 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:19:08.596636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7b396-4217-49e2-9cc4-970f2e46cbbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:08.597346 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7b396-4217-49e2-9cc4-970f2e46cbbf service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:08.597568 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7b396-4217-49e2-9cc4-970f2e46cbbf service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:08.597848 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7b396-4217-49e2-9cc4-970f2e46cbbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:08.599727 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-9ec7b396-4217-49e2-9cc4-970f2e46cbbf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:08.616530 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ddd862-11aa-46f6-8f20-88ea4bcdb6a0 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:08.619160 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:19:08.622718 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50023c78-4ab9-4133-8637-5a3d0020f046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:08.623345 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50023c78-4ab9-4133-8637-5a3d0020f046 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:08.623653 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50023c78-4ab9-4133-8637-5a3d0020f046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:08.623885 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50023c78-4ab9-4133-8637-5a3d0020f046 admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:08.624511 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50023c78-4ab9-4133-8637-5a3d0020f046 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:08.626428 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50023c78-4ab9-4133-8637-5a3d0020f046 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:08.649058 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1189/2385] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:19:08 2022] POST /identity/v3/users => generated 286 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:19:08.653064 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50023c78-4ab9-4133-8637-5a3d0020f046 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:08.663337 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec7b396-4217-49e2-9cc4-970f2e46cbbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:08.663838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50023c78-4ab9-4133-8637-5a3d0020f046 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:08.664862 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec7b396-4217-49e2-9cc4-970f2e46cbbf service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:08.669142 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233377b2-9f36-4395-b238-6cd6c6f515d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:08.669726 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233377b2-9f36-4395-b238-6cd6c6f515d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:08.670009 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233377b2-9f36-4395-b238-6cd6c6f515d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:08.670278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233377b2-9f36-4395-b238-6cd6c6f515d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:08.670944 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233377b2-9f36-4395-b238-6cd6c6f515d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:08.673131 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233377b2-9f36-4395-b238-6cd6c6f515d2 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:08.675085 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1197/2386] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:19:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:19:08.686045 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1190/2387] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:19:08.687515 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7b396-4217-49e2-9cc4-970f2e46cbbf service cinder] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:08.698385 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7b396-4217-49e2-9cc4-970f2e46cbbf service cinder] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:08.703457 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d986133-78b1-4caa-beb1-9d1746e8210b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:08.704062 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d986133-78b1-4caa-beb1-9d1746e8210b admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:08.704564 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d986133-78b1-4caa-beb1-9d1746e8210b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:08.704846 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d986133-78b1-4caa-beb1-9d1746e8210b admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:08.705497 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d986133-78b1-4caa-beb1-9d1746e8210b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:08.707087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d986133-78b1-4caa-beb1-9d1746e8210b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:08.717208 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1198/2388] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:19:08.722009 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233377b2-9f36-4395-b238-6cd6c6f515d2 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:08.732157 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233377b2-9f36-4395-b238-6cd6c6f515d2 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:08.737672 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5373af-c8ac-4261-a7b0-c90bef0219de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:08.738319 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5373af-c8ac-4261-a7b0-c90bef0219de admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:08.738597 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5373af-c8ac-4261-a7b0-c90bef0219de admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:08.738925 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5373af-c8ac-4261-a7b0-c90bef0219de admin admin] PATH_INFO: `/v3/projects/fd4a527e5cc441f5901536b58d4fe400/users/0956fadcbd1e4bcd853eaf7cd5115d93/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:08.756435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5373af-c8ac-4261-a7b0-c90bef0219de admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd4a527e5cc441f5901536b58d4fe400, user_id=0956fadcbd1e4bcd853eaf7cd5115d93, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:08.759079 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5373af-c8ac-4261-a7b0-c90bef0219de admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:08.773241 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1191/2389] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:19:08 2022] PUT /identity/v3/projects/fd4a527e5cc441f5901536b58d4fe400/users/0956fadcbd1e4bcd853eaf7cd5115d93/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:19:08.778383 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b3940-a0e9-4642-ad50-41187cb1dd87 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:08.778636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b3940-a0e9-4642-ad50-41187cb1dd87 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:08.778972 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b3940-a0e9-4642-ad50-41187cb1dd87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:08.798907 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-829b3940-a0e9-4642-ad50-41187cb1dd87 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:19:08.800117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-829b3940-a0e9-4642-ad50-41187cb1dd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:08.862907 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1199/2390] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:19:08.873294 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5373af-c8ac-4261-a7b0-c90bef0219de admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:08.875567 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-0b5373af-c8ac-4261-a7b0-c90bef0219de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:08.936855 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5373af-c8ac-4261-a7b0-c90bef0219de admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:08.940684 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aecaced-e132-44aa-ac92-989a1adb98fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:08.941296 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aecaced-e132-44aa-ac92-989a1adb98fa service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:08.941573 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aecaced-e132-44aa-ac92-989a1adb98fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:08.941838 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aecaced-e132-44aa-ac92-989a1adb98fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:08.951722 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aecaced-e132-44aa-ac92-989a1adb98fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:08.953658 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aecaced-e132-44aa-ac92-989a1adb98fa service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:08.965025 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1192/2391] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:19:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:19:13.188908 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022df1d3-2c3c-424a-8a08-5615d74534b1 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:13.189381 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022df1d3-2c3c-424a-8a08-5615d74534b1 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:13.189461 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022df1d3-2c3c-424a-8a08-5615d74534b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:13.238308 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-022df1d3-2c3c-424a-8a08-5615d74534b1 None None] MFA Rules not processed for user `0956fadcbd1e4bcd853eaf7cd5115d93`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:19:13.239461 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-022df1d3-2c3c-424a-8a08-5615d74534b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:13.249337 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aecaced-e132-44aa-ac92-989a1adb98fa service neutron] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:13.251512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-7aecaced-e132-44aa-ac92-989a1adb98fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:13.311192 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aecaced-e132-44aa-ac92-989a1adb98fa service neutron] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:13.315626 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1200/2392] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:19:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:19:13.316474 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39e5e27-e361-4d94-a68b-e36c51845616 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:13.317094 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e5e27-e361-4d94-a68b-e36c51845616 service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:13.317373 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e5e27-e361-4d94-a68b-e36c51845616 service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:13.317641 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e5e27-e361-4d94-a68b-e36c51845616 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:13.320066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022df1d3-2c3c-424a-8a08-5615d74534b1 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:13.330624 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:19:13.331109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022df1d3-2c3c-424a-8a08-5615d74534b1 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:13.352260 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c3564b-0a27-4588-9521-86762bdb8067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:13.353008 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3564b-0a27-4588-9521-86762bdb8067 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:13.353294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3564b-0a27-4588-9521-86762bdb8067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:13.353561 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3564b-0a27-4588-9521-86762bdb8067 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:13.354203 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c3564b-0a27-4588-9521-86762bdb8067 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:13.356126 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c3564b-0a27-4588-9521-86762bdb8067 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:13.402263 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39e5e27-e361-4d94-a68b-e36c51845616 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:13.405578 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1201/2393] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:19:13 2022] POST /identity/v3/projects => generated 371 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:19:13.405762 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39e5e27-e361-4d94-a68b-e36c51845616 service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:13.408683 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c3564b-0a27-4588-9521-86762bdb8067 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:13.418898 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1193/2394] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:19:13 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:19:13.433018 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:19:13.488741 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c3564b-0a27-4588-9521-86762bdb8067 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:13.495182 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af0256-e72e-4be2-b64e-5bbfc24cb3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:13.495802 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af0256-e72e-4be2-b64e-5bbfc24cb3bf admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:13.496082 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af0256-e72e-4be2-b64e-5bbfc24cb3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:13.496348 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af0256-e72e-4be2-b64e-5bbfc24cb3bf admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:13.496988 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af0256-e72e-4be2-b64e-5bbfc24cb3bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:13.499311 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af0256-e72e-4be2-b64e-5bbfc24cb3bf admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:13.524255 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1202/2395] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:19:13 2022] POST /identity/v3/users => generated 292 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:19:13.528337 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39e5e27-e361-4d94-a68b-e36c51845616 service cinder] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:13.539774 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39e5e27-e361-4d94-a68b-e36c51845616 service cinder] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:13.547109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08594b3b-4696-47ab-ad6e-26b142850aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:13.547892 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08594b3b-4696-47ab-ad6e-26b142850aab admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:13.548237 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08594b3b-4696-47ab-ad6e-26b142850aab admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:13.548579 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08594b3b-4696-47ab-ad6e-26b142850aab admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:13.549357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08594b3b-4696-47ab-ad6e-26b142850aab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:13.551058 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08594b3b-4696-47ab-ad6e-26b142850aab admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:13.563377 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1194/2396] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:19:13.567634 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af0256-e72e-4be2-b64e-5bbfc24cb3bf admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:13.578376 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af0256-e72e-4be2-b64e-5bbfc24cb3bf admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:13.583647 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b7b695-ec1c-43b3-a6e5-e85e34309458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:13.584494 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7b695-ec1c-43b3-a6e5-e85e34309458 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:13.584765 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7b695-ec1c-43b3-a6e5-e85e34309458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:13.585040 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7b695-ec1c-43b3-a6e5-e85e34309458 admin admin] PATH_INFO: `/v3/projects/f35e8d128cd44441b85c0ab5d5fe14cf/users/623f696b52574522a1fcab40de0f4764/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:13.603858 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b7b695-ec1c-43b3-a6e5-e85e34309458 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f35e8d128cd44441b85c0ab5d5fe14cf, user_id=623f696b52574522a1fcab40de0f4764, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:13.607231 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b7b695-ec1c-43b3-a6e5-e85e34309458 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:13.624085 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1203/2397] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:19:13 2022] PUT /identity/v3/projects/f35e8d128cd44441b85c0ab5d5fe14cf/users/623f696b52574522a1fcab40de0f4764/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:19:13.628097 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08594b3b-4696-47ab-ad6e-26b142850aab admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:13.639512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08594b3b-4696-47ab-ad6e-26b142850aab admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:13.715171 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b9f823-b151-493b-8490-63795143250a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:13.715854 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9f823-b151-493b-8490-63795143250a admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:13.716281 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9f823-b151-493b-8490-63795143250a admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:13.716498 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9f823-b151-493b-8490-63795143250a admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:13.717213 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b9f823-b151-493b-8490-63795143250a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:13.719597 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b9f823-b151-493b-8490-63795143250a admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:13.733516 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1195/2398] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:19:13.737640 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b7b695-ec1c-43b3-a6e5-e85e34309458 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:13.750131 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b7b695-ec1c-43b3-a6e5-e85e34309458 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:13.755956 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a0389-36c8-4b8a-934a-ad0003846328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:13.758879 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a0389-36c8-4b8a-934a-ad0003846328 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:13.758879 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a0389-36c8-4b8a-934a-ad0003846328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:13.758879 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a0389-36c8-4b8a-934a-ad0003846328 admin admin] PATH_INFO: `/v3/projects/f35e8d128cd44441b85c0ab5d5fe14cf/users/623f696b52574522a1fcab40de0f4764/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:13.759518 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55a0389-36c8-4b8a-934a-ad0003846328 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f35e8d128cd44441b85c0ab5d5fe14cf, user_id=623f696b52574522a1fcab40de0f4764, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:13.762197 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55a0389-36c8-4b8a-934a-ad0003846328 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:13.787057 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1204/2399] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:19:13 2022] PUT /identity/v3/projects/f35e8d128cd44441b85c0ab5d5fe14cf/users/623f696b52574522a1fcab40de0f4764/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:19:18.623604 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a743d-5ea4-44d7-a4d2-13a05aed3215 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:18.624135 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a743d-5ea4-44d7-a4d2-13a05aed3215 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:18.624947 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a743d-5ea4-44d7-a4d2-13a05aed3215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:18.643635 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a0389-36c8-4b8a-934a-ad0003846328 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:18.677335 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-e60a743d-5ea4-44d7-a4d2-13a05aed3215 None None] MFA Rules not processed for user `623f696b52574522a1fcab40de0f4764`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:19:18.678560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-e60a743d-5ea4-44d7-a4d2-13a05aed3215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:18.738312 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a0389-36c8-4b8a-934a-ad0003846328 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:18.752423 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed4e70-efc2-4ac6-9d3f-aa1d94c97cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:18.753646 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed4e70-efc2-4ac6-9d3f-aa1d94c97cdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:18.753947 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed4e70-efc2-4ac6-9d3f-aa1d94c97cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:18.754231 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed4e70-efc2-4ac6-9d3f-aa1d94c97cdb admin admin] PATH_INFO: `/v3/users/1fa5bf227bbc49f29fdc247d79f2583a` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:18.756840 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ed4e70-efc2-4ac6-9d3f-aa1d94c97cdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fa5bf227bbc49f29fdc247d79f2583a)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:18.759433 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ed4e70-efc2-4ac6-9d3f-aa1d94c97cdb admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:18.773743 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1196/2400] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:19:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:19:18.786928 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a743d-5ea4-44d7-a4d2-13a05aed3215 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:18.788693 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-e60a743d-5ea4-44d7-a4d2-13a05aed3215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:18.834534 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a4ed4e70-efc2-4ac6-9d3f-aa1d94c97cdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fa5bf227bbc49f29fdc247d79f2583a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:18.836094 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:19:18.847415 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a4ed4e70-efc2-4ac6-9d3f-aa1d94c97cdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fa5bf227bbc49f29fdc247d79f2583a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:18.848362 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a4ed4e70-efc2-4ac6-9d3f-aa1d94c97cdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fa5bf227bbc49f29fdc247d79f2583a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:18.849489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a4ed4e70-efc2-4ac6-9d3f-aa1d94c97cdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fa5bf227bbc49f29fdc247d79f2583a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:18.864242 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a4ed4e70-efc2-4ac6-9d3f-aa1d94c97cdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fa5bf227bbc49f29fdc247d79f2583a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:18.871495 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=85,keystone:INSERT=6,keystone:DELETE=9 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:19:18.891932 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1205/2401] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:19:18 2022] DELETE /identity/v3/users/1fa5bf227bbc49f29fdc247d79f2583a => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:19:18.896523 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed96350-b44e-42ba-857a-4efb8bf5b465 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:18.896778 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed96350-b44e-42ba-857a-4efb8bf5b465 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:18.897050 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed96350-b44e-42ba-857a-4efb8bf5b465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:18.916422 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a743d-5ea4-44d7-a4d2-13a05aed3215 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:18.932588 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-2ed96350-b44e-42ba-857a-4efb8bf5b465 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:19:18.934589 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-2ed96350-b44e-42ba-857a-4efb8bf5b465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:19.014594 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4359b7-825c-4b01-b0e3-7b2be8b4db1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:19.015406 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4359b7-825c-4b01-b0e3-7b2be8b4db1d service nova] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:19.015962 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4359b7-825c-4b01-b0e3-7b2be8b4db1d service nova] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:19.016304 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4359b7-825c-4b01-b0e3-7b2be8b4db1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:19.018742 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-ec4359b7-825c-4b01-b0e3-7b2be8b4db1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:19.026791 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1206/2402] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:19:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:19:19.035136 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed96350-b44e-42ba-857a-4efb8bf5b465 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:19.041852 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-2ed96350-b44e-42ba-857a-4efb8bf5b465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:19.101337 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4359b7-825c-4b01-b0e3-7b2be8b4db1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:19.103009 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4359b7-825c-4b01-b0e3-7b2be8b4db1d service nova] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:19.114817 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1197/2403] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:19:18 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:19:19.123482 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed96350-b44e-42ba-857a-4efb8bf5b465 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:19.127269 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba3005-05ee-45cc-817a-6fd0d720ab6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:19.127884 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba3005-05ee-45cc-817a-6fd0d720ab6d service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:19.128164 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba3005-05ee-45cc-817a-6fd0d720ab6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:19.128438 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba3005-05ee-45cc-817a-6fd0d720ab6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:19.137776 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daba3005-05ee-45cc-817a-6fd0d720ab6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:19.139280 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daba3005-05ee-45cc-817a-6fd0d720ab6d service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:19.149184 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1207/2404] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:19:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:19:19.446540 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4359b7-825c-4b01-b0e3-7b2be8b4db1d service nova] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:19.452134 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-ec4359b7-825c-4b01-b0e3-7b2be8b4db1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:19.470011 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4359b7-825c-4b01-b0e3-7b2be8b4db1d service nova] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:19.476358 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c2d96f-a19f-4432-83f8-28afb56ab0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:19.477390 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c2d96f-a19f-4432-83f8-28afb56ab0f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:19.477667 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c2d96f-a19f-4432-83f8-28afb56ab0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:19.477936 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c2d96f-a19f-4432-83f8-28afb56ab0f6 admin admin] PATH_INFO: `/v3/projects/2bc1c645366f4ecc82d65726ffac1c85` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:19.479099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c2d96f-a19f-4432-83f8-28afb56ab0f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bc1c645366f4ecc82d65726ffac1c85)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:19.480980 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c2d96f-a19f-4432-83f8-28afb56ab0f6 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:19.545625 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-61c2d96f-a19f-4432-83f8-28afb56ab0f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bc1c645366f4ecc82d65726ffac1c85', 'request_id': 'req-61c2d96f-a19f-4432-83f8-28afb56ab0f6'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:19.547121 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:19:19.554733 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-61c2d96f-a19f-4432-83f8-28afb56ab0f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bc1c645366f4ecc82d65726ffac1c85', 'request_id': 'req-61c2d96f-a19f-4432-83f8-28afb56ab0f6'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:19.564524 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-61c2d96f-a19f-4432-83f8-28afb56ab0f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bc1c645366f4ecc82d65726ffac1c85', 'request_id': 'req-61c2d96f-a19f-4432-83f8-28afb56ab0f6'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:19.578267 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-61c2d96f-a19f-4432-83f8-28afb56ab0f6 admin admin] The token cache is being invalidate because project 2bc1c645366f4ecc82d65726ffac1c85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:19:19.578549 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-61c2d96f-a19f-4432-83f8-28afb56ab0f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:19.579386 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-61c2d96f-a19f-4432-83f8-28afb56ab0f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:19.580835 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1198/2405] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:19:19 2022] DELETE /identity/v3/projects/2bc1c645366f4ecc82d65726ffac1c85 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:19:19.585671 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba3005-05ee-45cc-817a-6fd0d720ab6d service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:19.598087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba3005-05ee-45cc-817a-6fd0d720ab6d service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:19.670286 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced92c2-fbc9-4507-96cf-da7e4615dbe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:19.671017 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced92c2-fbc9-4507-96cf-da7e4615dbe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:19.671314 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced92c2-fbc9-4507-96cf-da7e4615dbe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:19.671586 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced92c2-fbc9-4507-96cf-da7e4615dbe6 admin admin] PATH_INFO: `/v3/users/05c76029555447baae6fa7a641e77c01` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:19.672832 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cced92c2-fbc9-4507-96cf-da7e4615dbe6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05c76029555447baae6fa7a641e77c01)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:19.675088 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cced92c2-fbc9-4507-96cf-da7e4615dbe6 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:19.723506 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:19:19.737635 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-cced92c2-fbc9-4507-96cf-da7e4615dbe6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05c76029555447baae6fa7a641e77c01'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:19.745774 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-cced92c2-fbc9-4507-96cf-da7e4615dbe6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05c76029555447baae6fa7a641e77c01'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:19.746535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-cced92c2-fbc9-4507-96cf-da7e4615dbe6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05c76029555447baae6fa7a641e77c01'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:19.747154 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-cced92c2-fbc9-4507-96cf-da7e4615dbe6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05c76029555447baae6fa7a641e77c01'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:19.754086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-cced92c2-fbc9-4507-96cf-da7e4615dbe6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05c76029555447baae6fa7a641e77c01'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:19.772690 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1208/2406] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:19:19 2022] DELETE /identity/v3/users/05c76029555447baae6fa7a641e77c01 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:19:20.125811 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c2d96f-a19f-4432-83f8-28afb56ab0f6 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:20.128785 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-61c2d96f-a19f-4432-83f8-28afb56ab0f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:20.164642 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c2d96f-a19f-4432-83f8-28afb56ab0f6 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:20.283076 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef48869-7640-4f4f-8f80-552d2ac8e646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:20.283730 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef48869-7640-4f4f-8f80-552d2ac8e646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:20.284086 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef48869-7640-4f4f-8f80-552d2ac8e646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:20.285279 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef48869-7640-4f4f-8f80-552d2ac8e646 admin admin] PATH_INFO: `/v3/projects/a25eca2687954bd795e628b1d373d309` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:20.298730 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef48869-7640-4f4f-8f80-552d2ac8e646 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a25eca2687954bd795e628b1d373d309)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:20.300729 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef48869-7640-4f4f-8f80-552d2ac8e646 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:20.412206 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-5ef48869-7640-4f4f-8f80-552d2ac8e646 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a25eca2687954bd795e628b1d373d309', 'request_id': 'req-5ef48869-7640-4f4f-8f80-552d2ac8e646'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:20.424678 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-5ef48869-7640-4f4f-8f80-552d2ac8e646 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a25eca2687954bd795e628b1d373d309', 'request_id': 'req-5ef48869-7640-4f4f-8f80-552d2ac8e646'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:20.435350 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-5ef48869-7640-4f4f-8f80-552d2ac8e646 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a25eca2687954bd795e628b1d373d309', 'request_id': 'req-5ef48869-7640-4f4f-8f80-552d2ac8e646'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:20.456993 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-5ef48869-7640-4f4f-8f80-552d2ac8e646 admin admin] The token cache is being invalidate because project a25eca2687954bd795e628b1d373d309 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:19:20.457301 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-5ef48869-7640-4f4f-8f80-552d2ac8e646 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:20.458018 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-5ef48869-7640-4f4f-8f80-552d2ac8e646 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:20.459648 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1199/2407] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:19:20 2022] DELETE /identity/v3/projects/a25eca2687954bd795e628b1d373d309 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:19:20.466436 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a39608-5081-4744-86f6-9d8fd404abbd None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:20.466961 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a39608-5081-4744-86f6-9d8fd404abbd None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:20.467315 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a39608-5081-4744-86f6-9d8fd404abbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:20.493707 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-d0a39608-5081-4744-86f6-9d8fd404abbd None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:19:20.495006 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d0a39608-5081-4744-86f6-9d8fd404abbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:20.513404 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4a83b-80dc-4de8-9a0f-3ab2e8c722b7 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:20.513687 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4a83b-80dc-4de8-9a0f-3ab2e8c722b7 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:20.513967 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4a83b-80dc-4de8-9a0f-3ab2e8c722b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:20.545513 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-d1e4a83b-80dc-4de8-9a0f-3ab2e8c722b7 None None] MFA Rules not processed for user `0956fadcbd1e4bcd853eaf7cd5115d93`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:19:20.547442 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-d1e4a83b-80dc-4de8-9a0f-3ab2e8c722b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:20.581750 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1209/2408] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:19:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:19:20.590615 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f67d3b-fb0d-4b61-9942-0329bd9376bb None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:20.590942 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f67d3b-fb0d-4b61-9942-0329bd9376bb None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:20.591217 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f67d3b-fb0d-4b61-9942-0329bd9376bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:20.635013 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-71f67d3b-fb0d-4b61-9942-0329bd9376bb None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:19:20.636589 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-71f67d3b-fb0d-4b61-9942-0329bd9376bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:20.643479 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1200/2409] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:19:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:19:20.651201 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1210/2410] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:19:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:19:20.656558 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4a83b-80dc-4de8-9a0f-3ab2e8c722b7 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:20.675165 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4a83b-80dc-4de8-9a0f-3ab2e8c722b7 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:20.687259 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e74263-731b-40f7-b7e6-307345bddf3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:20.688071 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e74263-731b-40f7-b7e6-307345bddf3c admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:20.688216 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e74263-731b-40f7-b7e6-307345bddf3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:20.688520 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e74263-731b-40f7-b7e6-307345bddf3c admin admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:20.689284 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e74263-731b-40f7-b7e6-307345bddf3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:19:20.689649 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e74263-731b-40f7-b7e6-307345bddf3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:20.691416 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e74263-731b-40f7-b7e6-307345bddf3c admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:20.703797 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:19:20.710994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1201/2411] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:19:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:19:20.712353 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f67d3b-fb0d-4b61-9942-0329bd9376bb None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:20.725619 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f67d3b-fb0d-4b61-9942-0329bd9376bb None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:20.731414 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac07c88-b4f8-4d67-b1ad-8d22e19bc695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:20.732229 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac07c88-b4f8-4d67-b1ad-8d22e19bc695 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:20.732617 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac07c88-b4f8-4d67-b1ad-8d22e19bc695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:20.732955 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac07c88-b4f8-4d67-b1ad-8d22e19bc695 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:20.733746 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac07c88-b4f8-4d67-b1ad-8d22e19bc695 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:20.736465 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac07c88-b4f8-4d67-b1ad-8d22e19bc695 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:20.779254 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1211/2412] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:19:20 2022] POST /identity/v3/projects => generated 373 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:19:20.783966 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e74263-731b-40f7-b7e6-307345bddf3c admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:20.799186 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e74263-731b-40f7-b7e6-307345bddf3c admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:20.887530 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d70f2-1381-488a-82ed-47ff639be53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:20.888162 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d70f2-1381-488a-82ed-47ff639be53f admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:20.888442 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d70f2-1381-488a-82ed-47ff639be53f admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:20.888713 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d70f2-1381-488a-82ed-47ff639be53f admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:20.889515 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d70f2-1381-488a-82ed-47ff639be53f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:20.891372 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d70f2-1381-488a-82ed-47ff639be53f admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:20.919363 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1202/2413] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:19:20 2022] POST /identity/v3/users => generated 287 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:19:20.924008 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac07c88-b4f8-4d67-b1ad-8d22e19bc695 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:20.939696 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac07c88-b4f8-4d67-b1ad-8d22e19bc695 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:20.949194 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d27dca-7b9d-4ee7-814b-f24aec028aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:20.949947 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d27dca-7b9d-4ee7-814b-f24aec028aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:20.950471 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d27dca-7b9d-4ee7-814b-f24aec028aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:20.950747 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d27dca-7b9d-4ee7-814b-f24aec028aa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:20.951589 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d27dca-7b9d-4ee7-814b-f24aec028aa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:20.953867 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d27dca-7b9d-4ee7-814b-f24aec028aa7 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:20.971683 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1212/2414] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:19:20 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:19:20.972896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d70f2-1381-488a-82ed-47ff639be53f admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:20.983496 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d70f2-1381-488a-82ed-47ff639be53f admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:20.989322 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd81d1a-7e37-430d-bba8-5b4aa55013eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:20.990178 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd81d1a-7e37-430d-bba8-5b4aa55013eb admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:20.990506 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd81d1a-7e37-430d-bba8-5b4aa55013eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:20.990729 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd81d1a-7e37-430d-bba8-5b4aa55013eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:20.992477 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd81d1a-7e37-430d-bba8-5b4aa55013eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:20.993255 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd81d1a-7e37-430d-bba8-5b4aa55013eb admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:21.021767 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1203/2415] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:19:20 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:19:21.026119 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d27dca-7b9d-4ee7-814b-f24aec028aa7 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:21.040523 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d27dca-7b9d-4ee7-814b-f24aec028aa7 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:21.048412 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baece9e-6c1b-49ae-b31d-fde08b69a199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:21.049251 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baece9e-6c1b-49ae-b31d-fde08b69a199 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:21.049373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baece9e-6c1b-49ae-b31d-fde08b69a199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:21.049993 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baece9e-6c1b-49ae-b31d-fde08b69a199 admin admin] PATH_INFO: `/v3/projects/493952e5240a4d5d953a3bb86d0f5e63/users/4364717b9fd643dfb7aad6cde84cb7e6/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:21.071820 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baece9e-6c1b-49ae-b31d-fde08b69a199 admin admin] RBAC: Authorizing `identity:create_grant(project_id=493952e5240a4d5d953a3bb86d0f5e63, user_id=4364717b9fd643dfb7aad6cde84cb7e6, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:21.075898 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baece9e-6c1b-49ae-b31d-fde08b69a199 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:21.108818 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1213/2416] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:19:21 2022] PUT /identity/v3/projects/493952e5240a4d5d953a3bb86d0f5e63/users/4364717b9fd643dfb7aad6cde84cb7e6/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:19:21.190999 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bef27-7e5a-4b69-a86e-53a75edb1ee0 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:21.191559 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bef27-7e5a-4b69-a86e-53a75edb1ee0 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:21.191559 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bef27-7e5a-4b69-a86e-53a75edb1ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:21.225760 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-309bef27-7e5a-4b69-a86e-53a75edb1ee0 None None] MFA Rules not processed for user `4364717b9fd643dfb7aad6cde84cb7e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:19:21.227041 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-309bef27-7e5a-4b69-a86e-53a75edb1ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:21.300843 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1204/2417] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:19:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:19:21.313132 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baece9e-6c1b-49ae-b31d-fde08b69a199 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:21.314921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-3baece9e-6c1b-49ae-b31d-fde08b69a199 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:21.367013 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:19:21.372485 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baece9e-6c1b-49ae-b31d-fde08b69a199 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:21.377992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9915624-b559-48af-9c58-3c79c4ee3626 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:21.379158 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9915624-b559-48af-9c58-3c79c4ee3626 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:21.379533 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9915624-b559-48af-9c58-3c79c4ee3626 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:21.379804 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9915624-b559-48af-9c58-3c79c4ee3626 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:21.390821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9915624-b559-48af-9c58-3c79c4ee3626 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:21.392418 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9915624-b559-48af-9c58-3c79c4ee3626 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:21.403063 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1214/2418] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:19:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:19:24.640531 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309bef27-7e5a-4b69-a86e-53a75edb1ee0 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:24.706903 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd127165-4a6f-47e6-ae5f-1ac48473fac7 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:24.708095 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd127165-4a6f-47e6-ae5f-1ac48473fac7 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:24.708693 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd127165-4a6f-47e6-ae5f-1ac48473fac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:24.741722 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-fd127165-4a6f-47e6-ae5f-1ac48473fac7 None None] MFA Rules not processed for user `1405e758a3af4e8ba659f498cd28ace1`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:19:24.744155 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-fd127165-4a6f-47e6-ae5f-1ac48473fac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:24.784771 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309bef27-7e5a-4b69-a86e-53a75edb1ee0 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:24.798589 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c05df-1cdc-4139-aebf-8681284efd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:24.799490 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c05df-1cdc-4139-aebf-8681284efd5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:24.799774 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c05df-1cdc-4139-aebf-8681284efd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:24.800147 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c05df-1cdc-4139-aebf-8681284efd5c admin admin] PATH_INFO: `/v3/users/4364717b9fd643dfb7aad6cde84cb7e6` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:24.801756 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54c05df-1cdc-4139-aebf-8681284efd5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4364717b9fd643dfb7aad6cde84cb7e6)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:24.805701 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54c05df-1cdc-4139-aebf-8681284efd5c admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:24.819651 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:19:24.870594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1215/2419] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Sep 12 14:19:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:19:24.888576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd127165-4a6f-47e6-ae5f-1ac48473fac7 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:24.890532 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-fd127165-4a6f-47e6-ae5f-1ac48473fac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:24.896612 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-c54c05df-1cdc-4139-aebf-8681284efd5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4364717b9fd643dfb7aad6cde84cb7e6'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:24.908029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-c54c05df-1cdc-4139-aebf-8681284efd5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4364717b9fd643dfb7aad6cde84cb7e6'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:24.909338 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-c54c05df-1cdc-4139-aebf-8681284efd5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4364717b9fd643dfb7aad6cde84cb7e6'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:24.910641 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-c54c05df-1cdc-4139-aebf-8681284efd5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4364717b9fd643dfb7aad6cde84cb7e6'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:24.926386 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-c54c05df-1cdc-4139-aebf-8681284efd5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4364717b9fd643dfb7aad6cde84cb7e6'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:24.954646 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1205/2420] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:19:24 2022] DELETE /identity/v3/users/4364717b9fd643dfb7aad6cde84cb7e6 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:19:24.958983 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5823eb1-31f3-4ba7-a765-d6275558fe24 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:24.959275 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5823eb1-31f3-4ba7-a765-d6275558fe24 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:24.959677 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5823eb1-31f3-4ba7-a765-d6275558fe24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:24.978864 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-f5823eb1-31f3-4ba7-a765-d6275558fe24 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:19:24.980075 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-f5823eb1-31f3-4ba7-a765-d6275558fe24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:24.993332 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd127165-4a6f-47e6-ae5f-1ac48473fac7 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:25.047069 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1206/2421] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:19:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:19:25.057007 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5823eb1-31f3-4ba7-a765-d6275558fe24 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:25.058408 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-f5823eb1-31f3-4ba7-a765-d6275558fe24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:25.062448 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d38a7d-4a84-4a5e-a719-4f5f4490bf74 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a87ea9c3c5409b97a35c4857ae3ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:25.063125 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d38a7d-4a84-4a5e-a719-4f5f4490bf74 service swift] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:25.063405 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d38a7d-4a84-4a5e-a719-4f5f4490bf74 service swift] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:25.063677 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d38a7d-4a84-4a5e-a719-4f5f4490bf74 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:25.065405 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d8d38a7d-4a84-4a5e-a719-4f5f4490bf74 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:25.106204 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:19:25.119412 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5823eb1-31f3-4ba7-a765-d6275558fe24 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:25.123404 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1099400-1f64-43e7-a77b-e0c47817e172 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:25.123585 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1099400-1f64-43e7-a77b-e0c47817e172 service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:25.123858 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1099400-1f64-43e7-a77b-e0c47817e172 service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:25.124663 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1099400-1f64-43e7-a77b-e0c47817e172 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:25.125967 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d38a7d-4a84-4a5e-a719-4f5f4490bf74 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:25.127648 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d38a7d-4a84-4a5e-a719-4f5f4490bf74 service swift] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:25.135725 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1099400-1f64-43e7-a77b-e0c47817e172 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:25.137155 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1099400-1f64-43e7-a77b-e0c47817e172 service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:25.138893 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1216/2422] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Sep 12 14:19:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 12 14:19:25.148883 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1207/2423] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:19:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:19:25.550648 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d38a7d-4a84-4a5e-a719-4f5f4490bf74 service swift] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:25.552322 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d8d38a7d-4a84-4a5e-a719-4f5f4490bf74 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:25.579868 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d38a7d-4a84-4a5e-a719-4f5f4490bf74 service swift] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:25.587021 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f07bb-8fa6-4cb3-a4a6-2070965d104c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:25.591928 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f07bb-8fa6-4cb3-a4a6-2070965d104c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:25.592180 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f07bb-8fa6-4cb3-a4a6-2070965d104c admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:25.592460 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f07bb-8fa6-4cb3-a4a6-2070965d104c admin admin] PATH_INFO: `/v3/projects/493952e5240a4d5d953a3bb86d0f5e63` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:25.594174 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f07bb-8fa6-4cb3-a4a6-2070965d104c admin admin] RBAC: Authorizing `identity:delete_project(project_id=493952e5240a4d5d953a3bb86d0f5e63)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:25.600005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f07bb-8fa6-4cb3-a4a6-2070965d104c admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:25.672736 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d18f07bb-8fa6-4cb3-a4a6-2070965d104c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '493952e5240a4d5d953a3bb86d0f5e63', 'request_id': 'req-d18f07bb-8fa6-4cb3-a4a6-2070965d104c'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:25.681831 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d18f07bb-8fa6-4cb3-a4a6-2070965d104c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '493952e5240a4d5d953a3bb86d0f5e63', 'request_id': 'req-d18f07bb-8fa6-4cb3-a4a6-2070965d104c'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:25.693869 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d18f07bb-8fa6-4cb3-a4a6-2070965d104c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '493952e5240a4d5d953a3bb86d0f5e63', 'request_id': 'req-d18f07bb-8fa6-4cb3-a4a6-2070965d104c'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:25.711721 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d18f07bb-8fa6-4cb3-a4a6-2070965d104c admin admin] The token cache is being invalidate because project 493952e5240a4d5d953a3bb86d0f5e63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:19:25.712412 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d18f07bb-8fa6-4cb3-a4a6-2070965d104c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:25.712888 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d18f07bb-8fa6-4cb3-a4a6-2070965d104c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:19:25.714655 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1217/2424] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:19:25 2022] DELETE /identity/v3/projects/493952e5240a4d5d953a3bb86d0f5e63 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:19:25.719699 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ed8f4-6e5a-4996-83ec-e7a9b2637919 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:25.720169 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ed8f4-6e5a-4996-83ec-e7a9b2637919 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:25.720482 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ed8f4-6e5a-4996-83ec-e7a9b2637919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:25.751908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-844ed8f4-6e5a-4996-83ec-e7a9b2637919 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:19:25.753158 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-844ed8f4-6e5a-4996-83ec-e7a9b2637919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:25.790353 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:19:25.878909 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1208/2425] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:19:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:19:25.885623 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d6042-9808-4c7b-8e32-9c5e638237d9 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:25.886077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d6042-9808-4c7b-8e32-9c5e638237d9 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:25.886532 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d6042-9808-4c7b-8e32-9c5e638237d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:25.921893 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-283d6042-9808-4c7b-8e32-9c5e638237d9 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:19:25.923560 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-283d6042-9808-4c7b-8e32-9c5e638237d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:25.936044 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1218/2426] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:19:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:19:25.941211 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ed8f4-6e5a-4996-83ec-e7a9b2637919 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:25.955048 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ed8f4-6e5a-4996-83ec-e7a9b2637919 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:25.965785 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed94d97-2879-4d48-a0ba-9caa421b725a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:25.966571 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed94d97-2879-4d48-a0ba-9caa421b725a admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:25.966983 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed94d97-2879-4d48-a0ba-9caa421b725a admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:25.967359 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed94d97-2879-4d48-a0ba-9caa421b725a admin admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:25.968683 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed94d97-2879-4d48-a0ba-9caa421b725a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:19:25.969081 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed94d97-2879-4d48-a0ba-9caa421b725a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:25.974901 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed94d97-2879-4d48-a0ba-9caa421b725a admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:25.994391 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1209/2427] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:19:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:19:25.998550 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d6042-9808-4c7b-8e32-9c5e638237d9 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:26.008648 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d6042-9808-4c7b-8e32-9c5e638237d9 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:26.016702 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357f620-14f1-4316-95b8-361894cae98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:26.017386 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357f620-14f1-4316-95b8-361894cae98b admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:26.017776 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357f620-14f1-4316-95b8-361894cae98b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:26.018116 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357f620-14f1-4316-95b8-361894cae98b admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:26.018902 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0357f620-14f1-4316-95b8-361894cae98b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:26.020952 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0357f620-14f1-4316-95b8-361894cae98b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:26.062366 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1219/2428] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:19:25 2022] POST /identity/v3/projects => generated 371 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:19:26.066315 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed94d97-2879-4d48-a0ba-9caa421b725a admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:26.085872 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed94d97-2879-4d48-a0ba-9caa421b725a admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:26.217763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bac35f-449e-43d6-a59c-481c38375e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:26.219075 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bac35f-449e-43d6-a59c-481c38375e23 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:26.219362 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bac35f-449e-43d6-a59c-481c38375e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:26.219640 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bac35f-449e-43d6-a59c-481c38375e23 admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:26.220335 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bac35f-449e-43d6-a59c-481c38375e23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:26.222418 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bac35f-449e-43d6-a59c-481c38375e23 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:26.266265 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1210/2429] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:19:26 2022] POST /identity/v3/users => generated 286 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:19:26.270009 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357f620-14f1-4316-95b8-361894cae98b admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:26.289467 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357f620-14f1-4316-95b8-361894cae98b admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:26.308694 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29218fd7-fc3d-4d47-a1b2-dc4a720e8029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:26.308694 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29218fd7-fc3d-4d47-a1b2-dc4a720e8029 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:26.308694 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29218fd7-fc3d-4d47-a1b2-dc4a720e8029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:26.308694 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29218fd7-fc3d-4d47-a1b2-dc4a720e8029 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:26.309122 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29218fd7-fc3d-4d47-a1b2-dc4a720e8029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:26.310464 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29218fd7-fc3d-4d47-a1b2-dc4a720e8029 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:26.325452 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1220/2430] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:19:26 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:19:26.333178 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bac35f-449e-43d6-a59c-481c38375e23 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:26.346133 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bac35f-449e-43d6-a59c-481c38375e23 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:26.352458 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632cced7-4f44-47b6-ad85-7410cd171b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:26.353082 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632cced7-4f44-47b6-ad85-7410cd171b99 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:26.353357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632cced7-4f44-47b6-ad85-7410cd171b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:26.353625 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632cced7-4f44-47b6-ad85-7410cd171b99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:26.354297 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632cced7-4f44-47b6-ad85-7410cd171b99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:26.356015 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632cced7-4f44-47b6-ad85-7410cd171b99 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:26.371469 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1211/2431] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:19:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:19:26.372939 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29218fd7-fc3d-4d47-a1b2-dc4a720e8029 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:26.384267 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29218fd7-fc3d-4d47-a1b2-dc4a720e8029 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:26.395206 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcaee57-0ccf-4208-9bac-2c09caf5d558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:26.397771 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcaee57-0ccf-4208-9bac-2c09caf5d558 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:26.398048 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcaee57-0ccf-4208-9bac-2c09caf5d558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:26.398345 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcaee57-0ccf-4208-9bac-2c09caf5d558 admin admin] PATH_INFO: `/v3/projects/886e4a96cfbf4f879d28a681dd348b55/users/13f3433daee04e8cac7a8b3c68ea57f2/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:26.415187 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcaee57-0ccf-4208-9bac-2c09caf5d558 admin admin] RBAC: Authorizing `identity:create_grant(project_id=886e4a96cfbf4f879d28a681dd348b55, user_id=13f3433daee04e8cac7a8b3c68ea57f2, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:26.417826 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcaee57-0ccf-4208-9bac-2c09caf5d558 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:26.437116 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1221/2432] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:19:26 2022] PUT /identity/v3/projects/886e4a96cfbf4f879d28a681dd348b55/users/13f3433daee04e8cac7a8b3c68ea57f2/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:19:26.448411 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127124b-d577-4f35-8b73-8f41a3c86fb0 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:26.448671 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127124b-d577-4f35-8b73-8f41a3c86fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:26.449072 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127124b-d577-4f35-8b73-8f41a3c86fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:26.483355 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-a127124b-d577-4f35-8b73-8f41a3c86fb0 None None] MFA Rules not processed for user `13f3433daee04e8cac7a8b3c68ea57f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:19:26.496618 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-a127124b-d577-4f35-8b73-8f41a3c86fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:26.615673 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1212/2433] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:19:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:19:31.286579 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcaee57-0ccf-4208-9bac-2c09caf5d558 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:31.288520 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ebcaee57-0ccf-4208-9bac-2c09caf5d558 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:31.373802 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcaee57-0ccf-4208-9bac-2c09caf5d558 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:31.378734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64b48bc-5e22-434f-b7ef-a3c31e79dd58 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2092a9639067461c9a837a891f56889b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:31.379385 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b48bc-5e22-434f-b7ef-a3c31e79dd58 service glance] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:31.379662 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b48bc-5e22-434f-b7ef-a3c31e79dd58 service glance] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:31.379930 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b48bc-5e22-434f-b7ef-a3c31e79dd58 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:31.392630 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64b48bc-5e22-434f-b7ef-a3c31e79dd58 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:31.394078 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64b48bc-5e22-434f-b7ef-a3c31e79dd58 service glance] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:31.399297 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:19:31.405110 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1222/2434] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:19:31 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:19:36.611115 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=85,keystone:INSERT=3 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:19:40.796115 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf196778-7f29-4fef-ac95-b6881ba0e0bf None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:40.796793 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf196778-7f29-4fef-ac95-b6881ba0e0bf None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:40.796793 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf196778-7f29-4fef-ac95-b6881ba0e0bf None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:40.798169 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1213/2435] 10.222.0.42 () {54 vars in 921 bytes} [Mon Sep 12 14:19:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:19:40.808596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbefde10-21d5-4856-90a3-f247dfb66e78 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:40.809405 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbefde10-21d5-4856-90a3-f247dfb66e78 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:40.809840 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbefde10-21d5-4856-90a3-f247dfb66e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:40.837658 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-bbefde10-21d5-4856-90a3-f247dfb66e78 None None] MFA Rules not processed for user `854ab2dcec5a48b18f409b33cb61b5da`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:19:40.839469 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-bbefde10-21d5-4856-90a3-f247dfb66e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:40.944700 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1223/2436] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Sep 12 14:19:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:19:40.957609 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf196778-7f29-4fef-ac95-b6881ba0e0bf None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:40.959663 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-bf196778-7f29-4fef-ac95-b6881ba0e0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:40.978213 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf196778-7f29-4fef-ac95-b6881ba0e0bf None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:40.982524 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208a13c-430d-4b1a-9313-f8738add56f4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2092a9639067461c9a837a891f56889b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:40.983217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208a13c-430d-4b1a-9313-f8738add56f4 service glance] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:40.983505 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208a13c-430d-4b1a-9313-f8738add56f4 service glance] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:40.983778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208a13c-430d-4b1a-9313-f8738add56f4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:40.996798 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8208a13c-430d-4b1a-9313-f8738add56f4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:40.999529 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8208a13c-430d-4b1a-9313-f8738add56f4 service glance] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:41.021205 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1214/2437] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:19:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:19:41.106973 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae5b1c9-ad6d-44a4-aa58-6bed9579522f None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:41.107339 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae5b1c9-ad6d-44a4-aa58-6bed9579522f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:41.107686 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae5b1c9-ad6d-44a4-aa58-6bed9579522f None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:41.109164 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1224/2438] 10.222.0.42 () {54 vars in 921 bytes} [Mon Sep 12 14:19:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:19:41.115125 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc26334-afc8-4564-b462-7ecfac8457f3 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:41.115406 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc26334-afc8-4564-b462-7ecfac8457f3 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:41.115979 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc26334-afc8-4564-b462-7ecfac8457f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:41.151815 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-bfc26334-afc8-4564-b462-7ecfac8457f3 None None] MFA Rules not processed for user `865b1647468c4efe8efe4febe2203a58`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:19:41.152656 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-bfc26334-afc8-4564-b462-7ecfac8457f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:41.222928 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1215/2439] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Sep 12 14:19:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:19:41.235313 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae5b1c9-ad6d-44a4-aa58-6bed9579522f None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:41.237109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-bae5b1c9-ad6d-44a4-aa58-6bed9579522f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:41.251503 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae5b1c9-ad6d-44a4-aa58-6bed9579522f None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:41.255127 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9861b4-db29-4b17-bb0e-7272c8ee5fdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:41.255757 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9861b4-db29-4b17-bb0e-7272c8ee5fdb service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:41.256040 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9861b4-db29-4b17-bb0e-7272c8ee5fdb service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:41.256314 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9861b4-db29-4b17-bb0e-7272c8ee5fdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:41.263863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9861b4-db29-4b17-bb0e-7272c8ee5fdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:41.265345 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9861b4-db29-4b17-bb0e-7272c8ee5fdb service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:41.274315 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1225/2440] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:19:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:19:41.361198 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa9967-5ea5-44b6-b7f0-3801cddc610b None None] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:41.361533 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa9967-5ea5-44b6-b7f0-3801cddc610b None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:41.361794 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa9967-5ea5-44b6-b7f0-3801cddc610b None None] PATH_INFO: `/` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:41.363600 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1216/2441] 10.222.0.42 () {54 vars in 921 bytes} [Mon Sep 12 14:19:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 14:19:41.369488 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff4aa1-42b1-4504-b2e5-21488f463013 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:41.369930 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff4aa1-42b1-4504-b2e5-21488f463013 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:41.370298 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff4aa1-42b1-4504-b2e5-21488f463013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:41.394085 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-e8ff4aa1-42b1-4504-b2e5-21488f463013 None None] MFA Rules not processed for user `854ab2dcec5a48b18f409b33cb61b5da`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:19:41.395365 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e8ff4aa1-42b1-4504-b2e5-21488f463013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:41.406121 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1226/2442] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Sep 12 14:19:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:19:41.418214 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa9967-5ea5-44b6-b7f0-3801cddc610b None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:41.432295 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa9967-5ea5-44b6-b7f0-3801cddc610b None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:41.437042 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4741477a-8813-4418-b225-7570b105b2ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:41.437749 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741477a-8813-4418-b225-7570b105b2ed service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:41.438029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741477a-8813-4418-b225-7570b105b2ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:41.438296 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741477a-8813-4418-b225-7570b105b2ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:41.474328 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4741477a-8813-4418-b225-7570b105b2ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:41.475899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4741477a-8813-4418-b225-7570b105b2ed service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:41.506351 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1217/2443] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:19:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:19:44.136420 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f64dd-447a-4b79-9641-0173ad0bfc3f None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:44.136892 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f64dd-447a-4b79-9641-0173ad0bfc3f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:44.137026 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f64dd-447a-4b79-9641-0173ad0bfc3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:44.156515 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-bd5f64dd-447a-4b79-9641-0173ad0bfc3f None None] MFA Rules not processed for user `1405e758a3af4e8ba659f498cd28ace1`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:19:44.157710 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-bd5f64dd-447a-4b79-9641-0173ad0bfc3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:44.213099 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1227/2444] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Sep 12 14:19:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:19:44.225550 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4741477a-8813-4418-b225-7570b105b2ed service neutron] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:19:44.226939 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-4741477a-8813-4418-b225-7570b105b2ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:19:44.295690 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4741477a-8813-4418-b225-7570b105b2ed service neutron] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:19:44.300645 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4cfc2b-4742-4d9a-a491-2070b2be2d81 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a87ea9c3c5409b97a35c4857ae3ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:19:44.301243 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4cfc2b-4742-4d9a-a491-2070b2be2d81 service swift] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:19:44.301521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4cfc2b-4742-4d9a-a491-2070b2be2d81 service swift] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:19:44.301792 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4cfc2b-4742-4d9a-a491-2070b2be2d81 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:19:44.309688 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4cfc2b-4742-4d9a-a491-2070b2be2d81 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:19:44.311562 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4cfc2b-4742-4d9a-a491-2070b2be2d81 service swift] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:19:44.319435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1218/2445] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Sep 12 14:19:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 14:19:54.206384 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=65 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:19:54.316235 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=67 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:20:00.079670 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f64dd-447a-4b79-9641-0173ad0bfc3f None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:00.139912 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f64dd-447a-4b79-9641-0173ad0bfc3f None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:00.144268 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f16203-5f40-4ea3-8403-2142468db8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:00.144879 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f16203-5f40-4ea3-8403-2142468db8f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:00.145152 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f16203-5f40-4ea3-8403-2142468db8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:00.145419 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f16203-5f40-4ea3-8403-2142468db8f6 admin admin] PATH_INFO: `/v3/users/13f3433daee04e8cac7a8b3c68ea57f2` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:00.146509 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f16203-5f40-4ea3-8403-2142468db8f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13f3433daee04e8cac7a8b3c68ea57f2)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:00.148420 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f16203-5f40-4ea3-8403-2142468db8f6 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:00.198421 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-21f16203-5f40-4ea3-8403-2142468db8f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13f3433daee04e8cac7a8b3c68ea57f2'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:00.207738 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-21f16203-5f40-4ea3-8403-2142468db8f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13f3433daee04e8cac7a8b3c68ea57f2'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:00.208416 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-21f16203-5f40-4ea3-8403-2142468db8f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13f3433daee04e8cac7a8b3c68ea57f2'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:00.209020 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-21f16203-5f40-4ea3-8403-2142468db8f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13f3433daee04e8cac7a8b3c68ea57f2'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:00.216430 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-21f16203-5f40-4ea3-8403-2142468db8f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13f3433daee04e8cac7a8b3c68ea57f2'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:00.235347 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1228/2446] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:20:00 2022] DELETE /identity/v3/users/13f3433daee04e8cac7a8b3c68ea57f2 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:20:00.239940 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c3770-1bbe-46e0-a1bc-1cd147818e7c None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:00.240434 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c3770-1bbe-46e0-a1bc-1cd147818e7c None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:00.240599 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c3770-1bbe-46e0-a1bc-1cd147818e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:00.259291 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-5f9c3770-1bbe-46e0-a1bc-1cd147818e7c None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:20:00.260580 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-5f9c3770-1bbe-46e0-a1bc-1cd147818e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:00.321585 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1219/2447] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:20:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:20:00.332432 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f16203-5f40-4ea3-8403-2142468db8f6 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:00.333856 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-21f16203-5f40-4ea3-8403-2142468db8f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:00.391610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f16203-5f40-4ea3-8403-2142468db8f6 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:00.395349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b05a2f-a868-480b-912c-28a53a1368b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:00.395949 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b05a2f-a868-480b-912c-28a53a1368b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:00.396223 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b05a2f-a868-480b-912c-28a53a1368b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:00.396490 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b05a2f-a868-480b-912c-28a53a1368b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:00.405352 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b05a2f-a868-480b-912c-28a53a1368b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:00.406747 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b05a2f-a868-480b-912c-28a53a1368b3 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:00.415844 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1229/2448] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:20:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:20:00.758451 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c3770-1bbe-46e0-a1bc-1cd147818e7c None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:00.759967 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-5f9c3770-1bbe-46e0-a1bc-1cd147818e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:00.772043 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c3770-1bbe-46e0-a1bc-1cd147818e7c None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:00.776907 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684b3f5-015a-4f80-a694-7491570cab1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:00.777512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684b3f5-015a-4f80-a694-7491570cab1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:00.777814 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684b3f5-015a-4f80-a694-7491570cab1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:00.778082 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684b3f5-015a-4f80-a694-7491570cab1d admin admin] PATH_INFO: `/v3/projects/886e4a96cfbf4f879d28a681dd348b55` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:00.779356 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684b3f5-015a-4f80-a694-7491570cab1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=886e4a96cfbf4f879d28a681dd348b55)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:00.781190 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684b3f5-015a-4f80-a694-7491570cab1d admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:00.828858 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-f684b3f5-015a-4f80-a694-7491570cab1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '886e4a96cfbf4f879d28a681dd348b55', 'request_id': 'req-f684b3f5-015a-4f80-a694-7491570cab1d'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:00.839268 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-f684b3f5-015a-4f80-a694-7491570cab1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '886e4a96cfbf4f879d28a681dd348b55', 'request_id': 'req-f684b3f5-015a-4f80-a694-7491570cab1d'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:00.846705 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-f684b3f5-015a-4f80-a694-7491570cab1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '886e4a96cfbf4f879d28a681dd348b55', 'request_id': 'req-f684b3f5-015a-4f80-a694-7491570cab1d'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:00.858857 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-f684b3f5-015a-4f80-a694-7491570cab1d admin admin] The token cache is being invalidate because project 886e4a96cfbf4f879d28a681dd348b55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:20:00.859163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-f684b3f5-015a-4f80-a694-7491570cab1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:00.859800 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-f684b3f5-015a-4f80-a694-7491570cab1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:00.861262 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1220/2449] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:20:00 2022] DELETE /identity/v3/projects/886e4a96cfbf4f879d28a681dd348b55 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:20:00.866053 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd1a34-4501-45ca-8d70-f8e1ac7bf177 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:00.866338 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd1a34-4501-45ca-8d70-f8e1ac7bf177 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:00.866607 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd1a34-4501-45ca-8d70-f8e1ac7bf177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:00.886191 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-84bd1a34-4501-45ca-8d70-f8e1ac7bf177 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:20:00.887406 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-84bd1a34-4501-45ca-8d70-f8e1ac7bf177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:00.903391 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:20:00.950086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1230/2450] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:20:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:20:00.957792 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4b927-4baf-4b18-9ad1-201626e39e11 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:00.958182 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4b927-4baf-4b18-9ad1-201626e39e11 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:00.958517 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4b927-4baf-4b18-9ad1-201626e39e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:00.977641 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-2bc4b927-4baf-4b18-9ad1-201626e39e11 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:20:00.978869 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-2bc4b927-4baf-4b18-9ad1-201626e39e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:00.987112 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1221/2451] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:20:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:20:00.990988 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd1a34-4501-45ca-8d70-f8e1ac7bf177 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:01.000978 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd1a34-4501-45ca-8d70-f8e1ac7bf177 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:01.005597 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8ae81-63fc-4269-a44d-06e07cf7ecd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:01.006196 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8ae81-63fc-4269-a44d-06e07cf7ecd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:01.006470 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8ae81-63fc-4269-a44d-06e07cf7ecd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:01.006736 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8ae81-63fc-4269-a44d-06e07cf7ecd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:01.007410 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf8ae81-63fc-4269-a44d-06e07cf7ecd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:20:01.007707 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf8ae81-63fc-4269-a44d-06e07cf7ecd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:01.009266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf8ae81-63fc-4269-a44d-06e07cf7ecd5 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:01.020734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1231/2452] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:20:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:20:01.024913 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc4b927-4baf-4b18-9ad1-201626e39e11 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:01.034510 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc4b927-4baf-4b18-9ad1-201626e39e11 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:01.039091 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07598ec1-5ed3-491d-b1ec-cfd7d444d305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:01.039669 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07598ec1-5ed3-491d-b1ec-cfd7d444d305 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:01.039941 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07598ec1-5ed3-491d-b1ec-cfd7d444d305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:01.040206 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07598ec1-5ed3-491d-b1ec-cfd7d444d305 admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:01.040857 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07598ec1-5ed3-491d-b1ec-cfd7d444d305 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:01.042722 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07598ec1-5ed3-491d-b1ec-cfd7d444d305 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:01.070936 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1222/2453] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:20:01 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:20:01.075123 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8ae81-63fc-4269-a44d-06e07cf7ecd5 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:01.084976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8ae81-63fc-4269-a44d-06e07cf7ecd5 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:01.140139 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dada33e3-6dfe-4e98-961d-1484abce77cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:01.140734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada33e3-6dfe-4e98-961d-1484abce77cd admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:01.141015 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada33e3-6dfe-4e98-961d-1484abce77cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:01.141285 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada33e3-6dfe-4e98-961d-1484abce77cd admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:01.141904 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dada33e3-6dfe-4e98-961d-1484abce77cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:01.143837 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dada33e3-6dfe-4e98-961d-1484abce77cd admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:01.164783 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1232/2454] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:20:01 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:20:01.168766 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07598ec1-5ed3-491d-b1ec-cfd7d444d305 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:01.179803 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07598ec1-5ed3-491d-b1ec-cfd7d444d305 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:01.184361 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a866aa7-d8c8-4d41-bccd-b3b765812edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:01.184976 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a866aa7-d8c8-4d41-bccd-b3b765812edb admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:01.185253 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a866aa7-d8c8-4d41-bccd-b3b765812edb admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:01.185520 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a866aa7-d8c8-4d41-bccd-b3b765812edb admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:01.186418 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a866aa7-d8c8-4d41-bccd-b3b765812edb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:01.188014 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a866aa7-d8c8-4d41-bccd-b3b765812edb admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:01.197907 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1223/2455] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:20:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:20:01.206998 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dada33e3-6dfe-4e98-961d-1484abce77cd admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:01.216983 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dada33e3-6dfe-4e98-961d-1484abce77cd admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:01.222087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b295f2-6360-4d27-84d9-e183246ceb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:01.222678 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b295f2-6360-4d27-84d9-e183246ceb4f admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:01.222997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b295f2-6360-4d27-84d9-e183246ceb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:01.223273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b295f2-6360-4d27-84d9-e183246ceb4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:01.223914 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b295f2-6360-4d27-84d9-e183246ceb4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:01.225495 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b295f2-6360-4d27-84d9-e183246ceb4f admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:01.236525 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1233/2456] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:20:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:20:01.240706 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a866aa7-d8c8-4d41-bccd-b3b765812edb admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:01.250439 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a866aa7-d8c8-4d41-bccd-b3b765812edb admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:01.254984 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a11e0-7de3-4fb0-a96f-87aca1d4a0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:01.255616 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a11e0-7de3-4fb0-a96f-87aca1d4a0f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:01.255892 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a11e0-7de3-4fb0-a96f-87aca1d4a0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:01.256161 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a11e0-7de3-4fb0-a96f-87aca1d4a0f9 admin admin] PATH_INFO: `/v3/projects/39a36ef424d746ad9ef385ad658c6ea0/users/ed72423aebfc4af1bc66d34fc820030d/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:01.270114 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a11e0-7de3-4fb0-a96f-87aca1d4a0f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39a36ef424d746ad9ef385ad658c6ea0, user_id=ed72423aebfc4af1bc66d34fc820030d, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:01.272748 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a11e0-7de3-4fb0-a96f-87aca1d4a0f9 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:01.285694 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1224/2457] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:20:01 2022] PUT /identity/v3/projects/39a36ef424d746ad9ef385ad658c6ea0/users/ed72423aebfc4af1bc66d34fc820030d/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:20:01.297569 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7e086-f279-4b5e-a6a4-e9e9df04bdd1 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:01.297851 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7e086-f279-4b5e-a6a4-e9e9df04bdd1 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:01.298121 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7e086-f279-4b5e-a6a4-e9e9df04bdd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:01.326258 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-37e7e086-f279-4b5e-a6a4-e9e9df04bdd1 None None] MFA Rules not processed for user `ed72423aebfc4af1bc66d34fc820030d`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:20:01.327661 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-37e7e086-f279-4b5e-a6a4-e9e9df04bdd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:01.390513 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1234/2458] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:20:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:20:01.401790 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a11e0-7de3-4fb0-a96f-87aca1d4a0f9 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:01.403483 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-0a4a11e0-7de3-4fb0-a96f-87aca1d4a0f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:01.426969 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:20:01.464591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a11e0-7de3-4fb0-a96f-87aca1d4a0f9 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:01.468934 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a89848f-8ad2-4217-a588-9ffd88c7efa0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:01.469533 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a89848f-8ad2-4217-a588-9ffd88c7efa0 service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:01.469808 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a89848f-8ad2-4217-a588-9ffd88c7efa0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:01.470077 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a89848f-8ad2-4217-a588-9ffd88c7efa0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:01.479200 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a89848f-8ad2-4217-a588-9ffd88c7efa0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:01.480619 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a89848f-8ad2-4217-a588-9ffd88c7efa0 service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:01.489401 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1225/2459] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:20:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:20:01.514262 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7e086-f279-4b5e-a6a4-e9e9df04bdd1 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:01.534999 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:20:01.579457 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7e086-f279-4b5e-a6a4-e9e9df04bdd1 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:01.584051 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d98d1c-a9df-405b-808d-45f6da3b2940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:01.584676 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d98d1c-a9df-405b-808d-45f6da3b2940 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:01.584957 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d98d1c-a9df-405b-808d-45f6da3b2940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:01.585229 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d98d1c-a9df-405b-808d-45f6da3b2940 admin admin] PATH_INFO: `/v3/users/ed72423aebfc4af1bc66d34fc820030d` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:01.586377 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d98d1c-a9df-405b-808d-45f6da3b2940 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed72423aebfc4af1bc66d34fc820030d)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:01.588310 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d98d1c-a9df-405b-808d-45f6da3b2940 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:01.637850 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-01d98d1c-a9df-405b-808d-45f6da3b2940 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed72423aebfc4af1bc66d34fc820030d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:01.646320 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-01d98d1c-a9df-405b-808d-45f6da3b2940 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed72423aebfc4af1bc66d34fc820030d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:01.647151 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-01d98d1c-a9df-405b-808d-45f6da3b2940 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed72423aebfc4af1bc66d34fc820030d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:01.647823 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-01d98d1c-a9df-405b-808d-45f6da3b2940 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed72423aebfc4af1bc66d34fc820030d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:01.656223 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-01d98d1c-a9df-405b-808d-45f6da3b2940 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed72423aebfc4af1bc66d34fc820030d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:01.673829 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1235/2460] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:20:01 2022] DELETE /identity/v3/users/ed72423aebfc4af1bc66d34fc820030d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:20:01.678319 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0db891-c3f4-42af-b50c-ce86d614b4b7 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:01.678542 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0db891-c3f4-42af-b50c-ce86d614b4b7 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:01.678817 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0db891-c3f4-42af-b50c-ce86d614b4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:01.699668 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-aa0db891-c3f4-42af-b50c-ce86d614b4b7 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:20:01.700847 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-aa0db891-c3f4-42af-b50c-ce86d614b4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:01.759983 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1226/2461] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:20:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:20:01.769918 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d98d1c-a9df-405b-808d-45f6da3b2940 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:01.771427 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-01d98d1c-a9df-405b-808d-45f6da3b2940 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:01.830629 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d98d1c-a9df-405b-808d-45f6da3b2940 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:01.838439 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b44641-6799-4e0a-a09d-cf76631f2444 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:01.839119 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b44641-6799-4e0a-a09d-cf76631f2444 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:01.839402 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b44641-6799-4e0a-a09d-cf76631f2444 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:01.839676 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b44641-6799-4e0a-a09d-cf76631f2444 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:01.849337 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b44641-6799-4e0a-a09d-cf76631f2444 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:01.850754 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b44641-6799-4e0a-a09d-cf76631f2444 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:01.860617 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1236/2462] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:20:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:20:02.151325 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0db891-c3f4-42af-b50c-ce86d614b4b7 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:02.152971 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-aa0db891-c3f4-42af-b50c-ce86d614b4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:02.165538 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0db891-c3f4-42af-b50c-ce86d614b4b7 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:02.170218 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfafae0-88aa-406d-aac0-2df59b95df4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:02.171674 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfafae0-88aa-406d-aac0-2df59b95df4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:02.172125 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfafae0-88aa-406d-aac0-2df59b95df4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:02.172463 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfafae0-88aa-406d-aac0-2df59b95df4d admin admin] PATH_INFO: `/v3/projects/39a36ef424d746ad9ef385ad658c6ea0` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:02.173655 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfafae0-88aa-406d-aac0-2df59b95df4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=39a36ef424d746ad9ef385ad658c6ea0)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:02.175687 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfafae0-88aa-406d-aac0-2df59b95df4d admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:02.221820 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-8dfafae0-88aa-406d-aac0-2df59b95df4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39a36ef424d746ad9ef385ad658c6ea0', 'request_id': 'req-8dfafae0-88aa-406d-aac0-2df59b95df4d'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:02.227324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-8dfafae0-88aa-406d-aac0-2df59b95df4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39a36ef424d746ad9ef385ad658c6ea0', 'request_id': 'req-8dfafae0-88aa-406d-aac0-2df59b95df4d'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:02.235116 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-8dfafae0-88aa-406d-aac0-2df59b95df4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39a36ef424d746ad9ef385ad658c6ea0', 'request_id': 'req-8dfafae0-88aa-406d-aac0-2df59b95df4d'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:02.245855 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-8dfafae0-88aa-406d-aac0-2df59b95df4d admin admin] The token cache is being invalidate because project 39a36ef424d746ad9ef385ad658c6ea0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:20:02.246462 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-8dfafae0-88aa-406d-aac0-2df59b95df4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:02.246822 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-8dfafae0-88aa-406d-aac0-2df59b95df4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:02.248350 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1227/2463] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:20:02 2022] DELETE /identity/v3/projects/39a36ef424d746ad9ef385ad658c6ea0 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:20:11.856661 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:20:12.242980 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:20:18.768765 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b44641-6799-4e0a-a09d-cf76631f2444 service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:18.770284 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-04b44641-6799-4e0a-a09d-cf76631f2444 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:18.887796 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b44641-6799-4e0a-a09d-cf76631f2444 service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:19.022666 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa46dd-560a-4095-844f-ce955a7192ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:19.023478 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa46dd-560a-4095-844f-ce955a7192ac service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:19.023834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa46dd-560a-4095-844f-ce955a7192ac service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:19.024172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa46dd-560a-4095-844f-ce955a7192ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:19.024382 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084739e-f6b7-4592-96b1-209bab022657 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:19.024832 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084739e-f6b7-4592-96b1-209bab022657 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:19.024985 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084739e-f6b7-4592-96b1-209bab022657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:19.026548 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-06fa46dd-560a-4095-844f-ce955a7192ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:19.043349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fa46dd-560a-4095-844f-ce955a7192ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:19.046065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fa46dd-560a-4095-844f-ce955a7192ac service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:19.049124 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-b084739e-f6b7-4592-96b1-209bab022657 None None] MFA Rules not processed for user `1405e758a3af4e8ba659f498cd28ace1`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:20:19.050611 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-b084739e-f6b7-4592-96b1-209bab022657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:19.061664 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1237/2464] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:20:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:20:19.115025 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1228/2465] 10.222.0.42 () {58 vars in 967 bytes} [Mon Sep 12 14:20:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:20:19.142806 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa46dd-560a-4095-844f-ce955a7192ac service nova] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:19.144221 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-06fa46dd-560a-4095-844f-ce955a7192ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:19.222303 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa46dd-560a-4095-844f-ce955a7192ac service nova] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:19.227963 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df833d0-a164-4cbb-8f31-f597654de114 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a87ea9c3c5409b97a35c4857ae3ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:19.228667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df833d0-a164-4cbb-8f31-f597654de114 service swift] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:19.229030 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df833d0-a164-4cbb-8f31-f597654de114 service swift] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:19.229367 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df833d0-a164-4cbb-8f31-f597654de114 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:19.239126 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df833d0-a164-4cbb-8f31-f597654de114 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:19.241728 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df833d0-a164-4cbb-8f31-f597654de114 service swift] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:19.252294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1238/2466] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Sep 12 14:20:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 14:20:19.993366 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b084739e-f6b7-4592-96b1-209bab022657 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:19.995047 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-b084739e-f6b7-4592-96b1-209bab022657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:20.061692 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b084739e-f6b7-4592-96b1-209bab022657 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:20.071705 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac8d7fc-ff3b-4080-941d-b67957f0079e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:20.072575 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8d7fc-ff3b-4080-941d-b67957f0079e service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:20.072990 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8d7fc-ff3b-4080-941d-b67957f0079e service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:20.073444 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8d7fc-ff3b-4080-941d-b67957f0079e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:20.076965 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-0ac8d7fc-ff3b-4080-941d-b67957f0079e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:20.178009 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac8d7fc-ff3b-4080-941d-b67957f0079e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:20.180057 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac8d7fc-ff3b-4080-941d-b67957f0079e service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:20.190797 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1229/2467] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:20:19 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:20:25.216113 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df833d0-a164-4cbb-8f31-f597654de114 service swift] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:25.225733 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df833d0-a164-4cbb-8f31-f597654de114 service swift] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:25.229346 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ab946-cde3-4834-bb3a-75902418bc84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:25.230024 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ab946-cde3-4834-bb3a-75902418bc84 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:25.230294 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ab946-cde3-4834-bb3a-75902418bc84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:25.230998 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ab946-cde3-4834-bb3a-75902418bc84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:25.232820 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-6b8ab946-cde3-4834-bb3a-75902418bc84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:25.243117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:20:25.296402 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8ab946-cde3-4834-bb3a-75902418bc84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:25.297813 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8ab946-cde3-4834-bb3a-75902418bc84 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:25.307527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1239/2468] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:20:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:20:26.328802 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac8d7fc-ff3b-4080-941d-b67957f0079e service neutron] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:26.330241 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-0ac8d7fc-ff3b-4080-941d-b67957f0079e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:26.342832 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac8d7fc-ff3b-4080-941d-b67957f0079e service neutron] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:26.346590 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62085049-0ffe-4926-b320-0d0b4a0f2b55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:26.347346 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62085049-0ffe-4926-b320-0d0b4a0f2b55 service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:26.347685 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62085049-0ffe-4926-b320-0d0b4a0f2b55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:26.347958 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62085049-0ffe-4926-b320-0d0b4a0f2b55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:26.349632 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-62085049-0ffe-4926-b320-0d0b4a0f2b55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:26.361713 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62085049-0ffe-4926-b320-0d0b4a0f2b55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:26.363085 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62085049-0ffe-4926-b320-0d0b4a0f2b55 service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:26.372775 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1230/2469] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:20:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:20:28.504187 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ab946-cde3-4834-bb3a-75902418bc84 service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:28.506227 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-6b8ab946-cde3-4834-bb3a-75902418bc84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:28.572188 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62085049-0ffe-4926-b320-0d0b4a0f2b55 service neutron] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:28.574491 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-62085049-0ffe-4926-b320-0d0b4a0f2b55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:28.603012 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62085049-0ffe-4926-b320-0d0b4a0f2b55 service neutron] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:28.618607 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60901865-9234-40b5-a703-0daf17ed8173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:28.620294 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60901865-9234-40b5-a703-0daf17ed8173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:28.623430 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60901865-9234-40b5-a703-0daf17ed8173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:28.623713 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60901865-9234-40b5-a703-0daf17ed8173 admin admin] PATH_INFO: `/v3/users/e7cb679ed52e42ca83a2084f1ad4bd92` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:28.625298 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60901865-9234-40b5-a703-0daf17ed8173 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7cb679ed52e42ca83a2084f1ad4bd92)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:28.630304 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60901865-9234-40b5-a703-0daf17ed8173 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:28.695706 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:20:28.697032 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ab946-cde3-4834-bb3a-75902418bc84 service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:28.706411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a18f907-8d20-4090-a945-5439240ac049 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce508bb9aa164a7fb1831a9b5ff5b932', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:28.707999 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18f907-8d20-4090-a945-5439240ac049 service placement] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:28.708535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18f907-8d20-4090-a945-5439240ac049 service placement] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:28.709334 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18f907-8d20-4090-a945-5439240ac049 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:28.714779 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-9a18f907-8d20-4090-a945-5439240ac049 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:28.716409 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-60901865-9234-40b5-a703-0daf17ed8173 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7cb679ed52e42ca83a2084f1ad4bd92'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:28.741697 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-60901865-9234-40b5-a703-0daf17ed8173 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7cb679ed52e42ca83a2084f1ad4bd92'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:28.744638 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-60901865-9234-40b5-a703-0daf17ed8173 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7cb679ed52e42ca83a2084f1ad4bd92'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:28.747175 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-60901865-9234-40b5-a703-0daf17ed8173 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7cb679ed52e42ca83a2084f1ad4bd92'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:28.748586 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a18f907-8d20-4090-a945-5439240ac049 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:28.750317 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a18f907-8d20-4090-a945-5439240ac049 service placement] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:28.758887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-60901865-9234-40b5-a703-0daf17ed8173 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7cb679ed52e42ca83a2084f1ad4bd92'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:28.765091 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1240/2470] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:20:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:20:28.791453 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1231/2471] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:20:28 2022] DELETE /identity/v3/users/e7cb679ed52e42ca83a2084f1ad4bd92 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:20:28.986392 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a18f907-8d20-4090-a945-5439240ac049 service placement] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:28.999833 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a18f907-8d20-4090-a945-5439240ac049 service placement] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:29.085707 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9400d-a576-440c-bde1-d7461aa81f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:29.086441 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9400d-a576-440c-bde1-d7461aa81f44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:29.086865 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9400d-a576-440c-bde1-d7461aa81f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:29.087374 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9400d-a576-440c-bde1-d7461aa81f44 admin admin] PATH_INFO: `/v3/projects/613c135e354b4262865198bd5c834cb1` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:29.088898 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9400d-a576-440c-bde1-d7461aa81f44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=613c135e354b4262865198bd5c834cb1)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:29.091064 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9400d-a576-440c-bde1-d7461aa81f44 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:29.167575 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:20:29.178210 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c7e9400d-a576-440c-bde1-d7461aa81f44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '613c135e354b4262865198bd5c834cb1', 'request_id': 'req-c7e9400d-a576-440c-bde1-d7461aa81f44'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:29.187047 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c7e9400d-a576-440c-bde1-d7461aa81f44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '613c135e354b4262865198bd5c834cb1', 'request_id': 'req-c7e9400d-a576-440c-bde1-d7461aa81f44'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:29.194184 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c7e9400d-a576-440c-bde1-d7461aa81f44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '613c135e354b4262865198bd5c834cb1', 'request_id': 'req-c7e9400d-a576-440c-bde1-d7461aa81f44'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:29.207997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c7e9400d-a576-440c-bde1-d7461aa81f44 admin admin] The token cache is being invalidate because project 613c135e354b4262865198bd5c834cb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:20:29.208347 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c7e9400d-a576-440c-bde1-d7461aa81f44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:29.209230 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c7e9400d-a576-440c-bde1-d7461aa81f44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:29.214936 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1241/2472] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:20:28 2022] DELETE /identity/v3/projects/613c135e354b4262865198bd5c834cb1 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:20:29.217340 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9065f57-b103-4826-a1ac-c1442b4bb7b0 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:29.217622 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9065f57-b103-4826-a1ac-c1442b4bb7b0 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:29.217892 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9065f57-b103-4826-a1ac-c1442b4bb7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:29.238533 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-f9065f57-b103-4826-a1ac-c1442b4bb7b0 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:20:29.239772 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-f9065f57-b103-4826-a1ac-c1442b4bb7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:29.299870 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1232/2473] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:20:29.310017 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e076b-59e2-490c-84e3-63c8794afac9 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:29.310440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e076b-59e2-490c-84e3-63c8794afac9 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:29.311192 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e076b-59e2-490c-84e3-63c8794afac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:29.333840 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-cc1e076b-59e2-490c-84e3-63c8794afac9 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:20:29.335104 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-cc1e076b-59e2-490c-84e3-63c8794afac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:29.344243 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1242/2474] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:20:29.348548 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9065f57-b103-4826-a1ac-c1442b4bb7b0 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:29.358876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9065f57-b103-4826-a1ac-c1442b4bb7b0 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:29.364183 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a77c626-ea0c-4cbe-ae9c-1e3dd2ac3237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:29.364915 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77c626-ea0c-4cbe-ae9c-1e3dd2ac3237 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:29.365193 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77c626-ea0c-4cbe-ae9c-1e3dd2ac3237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:29.365467 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77c626-ea0c-4cbe-ae9c-1e3dd2ac3237 admin admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:29.366109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a77c626-ea0c-4cbe-ae9c-1e3dd2ac3237 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:20:29.366411 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a77c626-ea0c-4cbe-ae9c-1e3dd2ac3237 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:29.368085 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a77c626-ea0c-4cbe-ae9c-1e3dd2ac3237 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:29.384012 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1233/2475] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:20:29.385167 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e076b-59e2-490c-84e3-63c8794afac9 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:29.395560 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e076b-59e2-490c-84e3-63c8794afac9 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:29.410765 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b46c38-7af3-43f9-90fc-f5dd70c308ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:29.411452 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b46c38-7af3-43f9-90fc-f5dd70c308ec admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:29.411897 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b46c38-7af3-43f9-90fc-f5dd70c308ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:29.412164 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b46c38-7af3-43f9-90fc-f5dd70c308ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:29.412882 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b46c38-7af3-43f9-90fc-f5dd70c308ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:29.414961 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b46c38-7af3-43f9-90fc-f5dd70c308ec admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:29.450608 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1243/2476] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:20:29 2022] POST /identity/v3/projects => generated 377 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:20:29.455189 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a77c626-ea0c-4cbe-ae9c-1e3dd2ac3237 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:29.469368 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a77c626-ea0c-4cbe-ae9c-1e3dd2ac3237 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:29.545423 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700f00e-f60d-4a34-a9d3-fda728cb64a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:29.546213 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700f00e-f60d-4a34-a9d3-fda728cb64a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:29.546498 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700f00e-f60d-4a34-a9d3-fda728cb64a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:29.546770 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700f00e-f60d-4a34-a9d3-fda728cb64a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:29.547451 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1700f00e-f60d-4a34-a9d3-fda728cb64a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:29.549407 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1700f00e-f60d-4a34-a9d3-fda728cb64a8 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:29.592322 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b46c38-7af3-43f9-90fc-f5dd70c308ec admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:29.592839 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1234/2477] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:20:29 2022] POST /identity/v3/users => generated 289 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:20:29.606155 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b46c38-7af3-43f9-90fc-f5dd70c308ec admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:29.614199 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a73684-c4cd-4034-8333-818829e84b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:29.614831 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a73684-c4cd-4034-8333-818829e84b4e admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:29.615244 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a73684-c4cd-4034-8333-818829e84b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:29.615508 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a73684-c4cd-4034-8333-818829e84b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:29.616169 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a73684-c4cd-4034-8333-818829e84b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:29.617966 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a73684-c4cd-4034-8333-818829e84b4e admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:29.632459 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1244/2478] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:20:29.636384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700f00e-f60d-4a34-a9d3-fda728cb64a8 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:29.648785 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700f00e-f60d-4a34-a9d3-fda728cb64a8 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:29.654060 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a915373-065b-4906-b806-3bd9be50bcaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:29.654679 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a915373-065b-4906-b806-3bd9be50bcaa admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:29.655152 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a915373-065b-4906-b806-3bd9be50bcaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:29.655583 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a915373-065b-4906-b806-3bd9be50bcaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:29.656386 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a915373-065b-4906-b806-3bd9be50bcaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:29.658492 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a915373-065b-4906-b806-3bd9be50bcaa admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:29.670355 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1235/2479] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:20:29.677043 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a73684-c4cd-4034-8333-818829e84b4e admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:29.697508 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a73684-c4cd-4034-8333-818829e84b4e admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:29.706435 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10987de4-946d-4d1e-a3f4-ecee02dc3b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:29.707663 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10987de4-946d-4d1e-a3f4-ecee02dc3b7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:29.708814 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10987de4-946d-4d1e-a3f4-ecee02dc3b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:29.709039 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10987de4-946d-4d1e-a3f4-ecee02dc3b7b admin admin] PATH_INFO: `/v3/projects/7dceba7df7bf4c1fad8656561cb63a1f/users/a64d6fad442b459791ae1988483d1970/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:29.739701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10987de4-946d-4d1e-a3f4-ecee02dc3b7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dceba7df7bf4c1fad8656561cb63a1f, user_id=a64d6fad442b459791ae1988483d1970, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:29.743139 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10987de4-946d-4d1e-a3f4-ecee02dc3b7b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:29.761664 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1245/2480] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:20:29 2022] PUT /identity/v3/projects/7dceba7df7bf4c1fad8656561cb63a1f/users/a64d6fad442b459791ae1988483d1970/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:20:29.775379 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f1b1a-dc59-4839-aa5a-2e16365d5a91 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:29.775635 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f1b1a-dc59-4839-aa5a-2e16365d5a91 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:29.775906 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f1b1a-dc59-4839-aa5a-2e16365d5a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:29.807299 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-7c0f1b1a-dc59-4839-aa5a-2e16365d5a91 None None] MFA Rules not processed for user `a64d6fad442b459791ae1988483d1970`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:20:29.808496 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-7c0f1b1a-dc59-4839-aa5a-2e16365d5a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:29.819321 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:20:29.918895 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1236/2481] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:20:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:20:29.923311 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10987de4-946d-4d1e-a3f4-ecee02dc3b7b admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:30.008656 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10987de4-946d-4d1e-a3f4-ecee02dc3b7b admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:30.016015 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df730da5-b103-452d-badf-d0c98a4830dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:30.016917 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df730da5-b103-452d-badf-d0c98a4830dd admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:30.017251 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df730da5-b103-452d-badf-d0c98a4830dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:30.017534 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df730da5-b103-452d-badf-d0c98a4830dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:30.018372 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df730da5-b103-452d-badf-d0c98a4830dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:30.020428 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df730da5-b103-452d-badf-d0c98a4830dd admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:30.028619 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f1b1a-dc59-4839-aa5a-2e16365d5a91 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:30.032014 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-7c0f1b1a-dc59-4839-aa5a-2e16365d5a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:30.049713 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f1b1a-dc59-4839-aa5a-2e16365d5a91 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:30.057440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1246/2482] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:20:29 2022] POST /identity/v3/projects => generated 377 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:20:30.057589 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eac74a-81f6-48d2-adc3-c5eab114729b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:30.057589 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eac74a-81f6-48d2-adc3-c5eab114729b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:30.057589 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eac74a-81f6-48d2-adc3-c5eab114729b admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:30.057589 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eac74a-81f6-48d2-adc3-c5eab114729b admin admin] PATH_INFO: `/v3/users/fff8e93029f447bb8294d9b516eb407f` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:30.057967 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eac74a-81f6-48d2-adc3-c5eab114729b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fff8e93029f447bb8294d9b516eb407f)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:30.058378 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df730da5-b103-452d-badf-d0c98a4830dd admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:30.059067 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eac74a-81f6-48d2-adc3-c5eab114729b admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:30.109037 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-33eac74a-81f6-48d2-adc3-c5eab114729b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fff8e93029f447bb8294d9b516eb407f'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:30.111202 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:20:30.118876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-33eac74a-81f6-48d2-adc3-c5eab114729b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fff8e93029f447bb8294d9b516eb407f'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:30.119612 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-33eac74a-81f6-48d2-adc3-c5eab114729b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fff8e93029f447bb8294d9b516eb407f'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:30.120266 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-33eac74a-81f6-48d2-adc3-c5eab114729b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fff8e93029f447bb8294d9b516eb407f'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:30.127768 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-33eac74a-81f6-48d2-adc3-c5eab114729b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fff8e93029f447bb8294d9b516eb407f'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:30.128540 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df730da5-b103-452d-badf-d0c98a4830dd admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:30.133447 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac81ca-fe4c-4685-a093-ca82b592c117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:30.134055 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac81ca-fe4c-4685-a093-ca82b592c117 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:30.134337 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac81ca-fe4c-4685-a093-ca82b592c117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:30.134611 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac81ca-fe4c-4685-a093-ca82b592c117 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:30.135284 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deac81ca-fe4c-4685-a093-ca82b592c117 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:30.137182 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deac81ca-fe4c-4685-a093-ca82b592c117 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:30.146496 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1237/2483] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:20:30 2022] DELETE /identity/v3/users/fff8e93029f447bb8294d9b516eb407f => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:20:30.163643 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1247/2484] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:20:30 2022] POST /identity/v3/users => generated 295 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:20:30.167623 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eac74a-81f6-48d2-adc3-c5eab114729b admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:30.169218 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-33eac74a-81f6-48d2-adc3-c5eab114729b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:30.244592 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eac74a-81f6-48d2-adc3-c5eab114729b admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:30.249298 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce3643e-70f6-450f-bc8b-6a436fb97215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:30.249933 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3643e-70f6-450f-bc8b-6a436fb97215 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:30.250192 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3643e-70f6-450f-bc8b-6a436fb97215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:30.250466 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3643e-70f6-450f-bc8b-6a436fb97215 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:30.251126 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce3643e-70f6-450f-bc8b-6a436fb97215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:30.252722 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce3643e-70f6-450f-bc8b-6a436fb97215 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:30.262663 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1238/2485] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:20:30.267468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac81ca-fe4c-4685-a093-ca82b592c117 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:30.277864 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac81ca-fe4c-4685-a093-ca82b592c117 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:30.283152 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f88cb-9e5a-41f5-903e-bb5a2ef64a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:30.283813 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f88cb-9e5a-41f5-903e-bb5a2ef64a20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:30.284092 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f88cb-9e5a-41f5-903e-bb5a2ef64a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:30.284368 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f88cb-9e5a-41f5-903e-bb5a2ef64a20 admin admin] PATH_INFO: `/v3/projects/3dc3326fe370446d8f941d4f2aaa411f/users/89383f692c824ef89d3c118fd82798aa/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:30.303180 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f88cb-9e5a-41f5-903e-bb5a2ef64a20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dc3326fe370446d8f941d4f2aaa411f, user_id=89383f692c824ef89d3c118fd82798aa, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:30.304047 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f88cb-9e5a-41f5-903e-bb5a2ef64a20 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:30.317921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1248/2486] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:20:30 2022] PUT /identity/v3/projects/3dc3326fe370446d8f941d4f2aaa411f/users/89383f692c824ef89d3c118fd82798aa/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:20:30.324112 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce3643e-70f6-450f-bc8b-6a436fb97215 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:30.324890 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f88cb-9e5a-41f5-903e-bb5a2ef64a20 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:30.327828 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-224f88cb-9e5a-41f5-903e-bb5a2ef64a20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:30.334964 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce3643e-70f6-450f-bc8b-6a436fb97215 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:30.359882 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:20:30.404468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f88cb-9e5a-41f5-903e-bb5a2ef64a20 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:30.407398 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0b10a-cc3d-40cb-8fcd-1f04877ca777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:30.408124 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0b10a-cc3d-40cb-8fcd-1f04877ca777 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:30.408413 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0b10a-cc3d-40cb-8fcd-1f04877ca777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:30.408678 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0b10a-cc3d-40cb-8fcd-1f04877ca777 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:30.409319 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0b10a-cc3d-40cb-8fcd-1f04877ca777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:30.409549 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211b450-a838-4c6e-9ef6-462328854b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:30.410178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211b450-a838-4c6e-9ef6-462328854b0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:30.410464 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211b450-a838-4c6e-9ef6-462328854b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:30.410742 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211b450-a838-4c6e-9ef6-462328854b0d admin admin] PATH_INFO: `/v3/projects/4a72919a47034a2684973f31e4217f3e` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:30.410920 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0b10a-cc3d-40cb-8fcd-1f04877ca777 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:30.411905 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211b450-a838-4c6e-9ef6-462328854b0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a72919a47034a2684973f31e4217f3e)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:30.414207 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211b450-a838-4c6e-9ef6-462328854b0d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:30.421145 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1239/2487] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:20:30.426032 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0b10a-cc3d-40cb-8fcd-1f04877ca777 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:30.436302 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0b10a-cc3d-40cb-8fcd-1f04877ca777 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:30.441221 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442c0a1c-1bd3-468e-8f16-a8b1d20b0f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:30.441860 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c0a1c-1bd3-468e-8f16-a8b1d20b0f53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:30.442135 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c0a1c-1bd3-468e-8f16-a8b1d20b0f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:30.442404 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c0a1c-1bd3-468e-8f16-a8b1d20b0f53 admin admin] PATH_INFO: `/v3/projects/3dc3326fe370446d8f941d4f2aaa411f/users/89383f692c824ef89d3c118fd82798aa/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:30.444558 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442c0a1c-1bd3-468e-8f16-a8b1d20b0f53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dc3326fe370446d8f941d4f2aaa411f, user_id=89383f692c824ef89d3c118fd82798aa, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:30.447766 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442c0a1c-1bd3-468e-8f16-a8b1d20b0f53 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:30.459775 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1240/2488] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:20:30 2022] PUT /identity/v3/projects/3dc3326fe370446d8f941d4f2aaa411f/users/89383f692c824ef89d3c118fd82798aa/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:20:30.468687 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c211b450-a838-4c6e-9ef6-462328854b0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a72919a47034a2684973f31e4217f3e', 'request_id': 'req-c211b450-a838-4c6e-9ef6-462328854b0d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:30.469996 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6aa9aa-0e05-40b8-bd95-6206f147b8a3 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:30.470298 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6aa9aa-0e05-40b8-bd95-6206f147b8a3 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:30.470593 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6aa9aa-0e05-40b8-bd95-6206f147b8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:30.472962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c211b450-a838-4c6e-9ef6-462328854b0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a72919a47034a2684973f31e4217f3e', 'request_id': 'req-c211b450-a838-4c6e-9ef6-462328854b0d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:30.479188 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c211b450-a838-4c6e-9ef6-462328854b0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a72919a47034a2684973f31e4217f3e', 'request_id': 'req-c211b450-a838-4c6e-9ef6-462328854b0d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:30.490327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c211b450-a838-4c6e-9ef6-462328854b0d admin admin] The token cache is being invalidate because project 4a72919a47034a2684973f31e4217f3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:20:30.490623 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c211b450-a838-4c6e-9ef6-462328854b0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:30.491328 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c211b450-a838-4c6e-9ef6-462328854b0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:30.492824 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1249/2489] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:20:30 2022] DELETE /identity/v3/projects/4a72919a47034a2684973f31e4217f3e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:20:30.497987 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c089f-6b01-4d8b-9a49-f1f08141694d None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:30.498272 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c089f-6b01-4d8b-9a49-f1f08141694d None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:30.498548 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c089f-6b01-4d8b-9a49-f1f08141694d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:30.501620 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-cd6aa9aa-0e05-40b8-bd95-6206f147b8a3 None None] MFA Rules not processed for user `89383f692c824ef89d3c118fd82798aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:20:30.502863 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-cd6aa9aa-0e05-40b8-bd95-6206f147b8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:30.580584 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1241/2490] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:20:30 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:20:30.591862 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6aa9aa-0e05-40b8-bd95-6206f147b8a3 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:30.593305 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-cd6aa9aa-0e05-40b8-bd95-6206f147b8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:30.659927 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-ce0c089f-6b01-4d8b-9a49-f1f08141694d None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:20:30.661169 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ce0c089f-6b01-4d8b-9a49-f1f08141694d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:30.672164 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6aa9aa-0e05-40b8-bd95-6206f147b8a3 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:30.676556 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bb0b1c-ce2f-4c66-a4b8-b382693253c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:30.677695 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb0b1c-ce2f-4c66-a4b8-b382693253c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:30.678079 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb0b1c-ce2f-4c66-a4b8-b382693253c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:30.678576 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb0b1c-ce2f-4c66-a4b8-b382693253c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:30.681059 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-30bb0b1c-ce2f-4c66-a4b8-b382693253c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:30.730093 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:20:30.734356 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:20:30.756908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bb0b1c-ce2f-4c66-a4b8-b382693253c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:30.758443 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bb0b1c-ce2f-4c66-a4b8-b382693253c4 service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:30.765266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1250/2491] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:20:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:20:30.772681 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe393c-9f41-4108-a9b9-879216038179 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:30.773204 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe393c-9f41-4108-a9b9-879216038179 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:30.773646 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe393c-9f41-4108-a9b9-879216038179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:30.780928 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1242/2492] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:20:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:20:30.796210 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-d0fe393c-9f41-4108-a9b9-879216038179 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:20:30.797491 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d0fe393c-9f41-4108-a9b9-879216038179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:30.805889 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1251/2493] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:20:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:20:30.811917 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bb0b1c-ce2f-4c66-a4b8-b382693253c4 service cinder] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:30.831913 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bb0b1c-ce2f-4c66-a4b8-b382693253c4 service cinder] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:30.842375 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15672a4-5ee4-4913-892a-18a5f1549239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:30.843184 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15672a4-5ee4-4913-892a-18a5f1549239 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:30.843656 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15672a4-5ee4-4913-892a-18a5f1549239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:30.844047 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15672a4-5ee4-4913-892a-18a5f1549239 admin admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:30.844841 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15672a4-5ee4-4913-892a-18a5f1549239 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:20:30.845255 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15672a4-5ee4-4913-892a-18a5f1549239 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:30.847174 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15672a4-5ee4-4913-892a-18a5f1549239 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:30.861517 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1243/2494] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:20:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:20:30.865884 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe393c-9f41-4108-a9b9-879216038179 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:30.875943 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe393c-9f41-4108-a9b9-879216038179 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:30.880874 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b43937-8d4c-45d3-a3cc-0739f1405165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:30.881579 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b43937-8d4c-45d3-a3cc-0739f1405165 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:30.881955 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b43937-8d4c-45d3-a3cc-0739f1405165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:30.882322 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b43937-8d4c-45d3-a3cc-0739f1405165 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:30.883315 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b43937-8d4c-45d3-a3cc-0739f1405165 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:30.885519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b43937-8d4c-45d3-a3cc-0739f1405165 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:30.919076 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1252/2495] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:20:30 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:20:30.924007 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15672a4-5ee4-4913-892a-18a5f1549239 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:30.937679 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15672a4-5ee4-4913-892a-18a5f1549239 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:31.009456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9b2f9-c2f1-4312-8c51-428d6d2e2978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:31.010166 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9b2f9-c2f1-4312-8c51-428d6d2e2978 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:31.010424 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9b2f9-c2f1-4312-8c51-428d6d2e2978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:31.010800 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9b2f9-c2f1-4312-8c51-428d6d2e2978 admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:31.011515 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e9b2f9-c2f1-4312-8c51-428d6d2e2978 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:31.014521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e9b2f9-c2f1-4312-8c51-428d6d2e2978 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:31.038469 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1244/2496] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:20:30 2022] POST /identity/v3/users => generated 290 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:20:31.043137 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b43937-8d4c-45d3-a3cc-0739f1405165 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:31.056388 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b43937-8d4c-45d3-a3cc-0739f1405165 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:31.063741 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa542f3d-9481-44f8-9190-e3b8df9f1ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:31.064502 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa542f3d-9481-44f8-9190-e3b8df9f1ea2 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:31.064936 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa542f3d-9481-44f8-9190-e3b8df9f1ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:31.065353 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa542f3d-9481-44f8-9190-e3b8df9f1ea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:31.066780 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa542f3d-9481-44f8-9190-e3b8df9f1ea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:31.069155 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa542f3d-9481-44f8-9190-e3b8df9f1ea2 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:31.081680 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1253/2497] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:20:31.085385 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9b2f9-c2f1-4312-8c51-428d6d2e2978 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:31.096945 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9b2f9-c2f1-4312-8c51-428d6d2e2978 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:31.102352 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb7893a-ed0f-4cbd-99e1-91ec7e4c3594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:31.102993 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7893a-ed0f-4cbd-99e1-91ec7e4c3594 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:31.103308 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7893a-ed0f-4cbd-99e1-91ec7e4c3594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:31.103626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7893a-ed0f-4cbd-99e1-91ec7e4c3594 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:31.104149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb7893a-ed0f-4cbd-99e1-91ec7e4c3594 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:31.105806 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb7893a-ed0f-4cbd-99e1-91ec7e4c3594 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:31.118886 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1245/2498] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:20:31.124351 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa542f3d-9481-44f8-9190-e3b8df9f1ea2 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:31.135260 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa542f3d-9481-44f8-9190-e3b8df9f1ea2 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:31.140859 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157e914d-a9b9-49e4-bfad-d9e8efd981df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:31.141577 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e914d-a9b9-49e4-bfad-d9e8efd981df admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:31.141879 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e914d-a9b9-49e4-bfad-d9e8efd981df admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:31.142164 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e914d-a9b9-49e4-bfad-d9e8efd981df admin admin] PATH_INFO: `/v3/projects/a0f854317e924e6b8317a35f1e3d038f/users/e62c30cf3c6244a6a750f0339eccaf5f/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:31.156972 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157e914d-a9b9-49e4-bfad-d9e8efd981df admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0f854317e924e6b8317a35f1e3d038f, user_id=e62c30cf3c6244a6a750f0339eccaf5f, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:31.159605 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157e914d-a9b9-49e4-bfad-d9e8efd981df admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:31.172516 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1254/2499] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:20:31 2022] PUT /identity/v3/projects/a0f854317e924e6b8317a35f1e3d038f/users/e62c30cf3c6244a6a750f0339eccaf5f/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:20:31.182870 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa9be1-b53e-4f68-8373-31e4e17c612e None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:31.183160 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa9be1-b53e-4f68-8373-31e4e17c612e None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:31.183430 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa9be1-b53e-4f68-8373-31e4e17c612e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:31.211291 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-a1fa9be1-b53e-4f68-8373-31e4e17c612e None None] MFA Rules not processed for user `e62c30cf3c6244a6a750f0339eccaf5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:20:31.212549 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-a1fa9be1-b53e-4f68-8373-31e4e17c612e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:31.276206 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1246/2500] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:20:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:20:31.280692 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157e914d-a9b9-49e4-bfad-d9e8efd981df admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:31.341131 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157e914d-a9b9-49e4-bfad-d9e8efd981df admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:31.345619 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315efece-6a36-43f5-8a71-285080323049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:31.346210 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315efece-6a36-43f5-8a71-285080323049 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:31.346485 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315efece-6a36-43f5-8a71-285080323049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:31.346749 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315efece-6a36-43f5-8a71-285080323049 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:31.347398 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315efece-6a36-43f5-8a71-285080323049 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:31.349249 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315efece-6a36-43f5-8a71-285080323049 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:31.376485 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1255/2501] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:20:31 2022] POST /identity/v3/projects => generated 379 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:20:31.380770 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa9be1-b53e-4f68-8373-31e4e17c612e None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:31.392495 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa9be1-b53e-4f68-8373-31e4e17c612e None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:31.425642 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:20:31.450950 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dda298-201d-4491-b7a6-a0e93167f296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:31.451551 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dda298-201d-4491-b7a6-a0e93167f296 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:31.451827 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dda298-201d-4491-b7a6-a0e93167f296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:31.452096 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dda298-201d-4491-b7a6-a0e93167f296 admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:31.452727 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dda298-201d-4491-b7a6-a0e93167f296 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:31.454619 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dda298-201d-4491-b7a6-a0e93167f296 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:31.477815 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1247/2502] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:20:31 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:20:31.478971 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315efece-6a36-43f5-8a71-285080323049 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:31.489395 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315efece-6a36-43f5-8a71-285080323049 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:31.493924 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcae6d1-eed6-405a-971c-4f77bdf189f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:31.494509 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcae6d1-eed6-405a-971c-4f77bdf189f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:31.494784 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcae6d1-eed6-405a-971c-4f77bdf189f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:31.495086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcae6d1-eed6-405a-971c-4f77bdf189f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:31.495719 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcae6d1-eed6-405a-971c-4f77bdf189f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:31.497282 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcae6d1-eed6-405a-971c-4f77bdf189f4 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:31.507372 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1256/2503] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:20:31.511368 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dda298-201d-4491-b7a6-a0e93167f296 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:31.521276 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dda298-201d-4491-b7a6-a0e93167f296 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:31.525880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f199f3b-d615-4377-a910-9a8589e3c643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:31.526522 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f199f3b-d615-4377-a910-9a8589e3c643 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:31.526802 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f199f3b-d615-4377-a910-9a8589e3c643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:31.527107 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f199f3b-d615-4377-a910-9a8589e3c643 admin admin] PATH_INFO: `/v3/projects/26d47d43237f4b31bf2ce9e116b348f4/users/b83b9f5d4e2b455698fbf6ecee70cbda/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:31.541606 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f199f3b-d615-4377-a910-9a8589e3c643 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26d47d43237f4b31bf2ce9e116b348f4, user_id=b83b9f5d4e2b455698fbf6ecee70cbda, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:31.544269 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f199f3b-d615-4377-a910-9a8589e3c643 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:31.557149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1248/2504] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:20:31 2022] PUT /identity/v3/projects/26d47d43237f4b31bf2ce9e116b348f4/users/b83b9f5d4e2b455698fbf6ecee70cbda/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:20:31.562003 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcae6d1-eed6-405a-971c-4f77bdf189f4 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:31.571574 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcae6d1-eed6-405a-971c-4f77bdf189f4 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:31.620929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:20:31.630203 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdbdbbc-122f-4700-96b8-4a59bc3ff1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:31.630808 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbdbbc-122f-4700-96b8-4a59bc3ff1d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:31.631118 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbdbbc-122f-4700-96b8-4a59bc3ff1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:31.631387 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbdbbc-122f-4700-96b8-4a59bc3ff1d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:31.632021 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdbdbbc-122f-4700-96b8-4a59bc3ff1d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:31.633611 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdbdbbc-122f-4700-96b8-4a59bc3ff1d3 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:31.643265 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1257/2505] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:20:31.647418 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f199f3b-d615-4377-a910-9a8589e3c643 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:31.657542 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f199f3b-d615-4377-a910-9a8589e3c643 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:31.662011 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6939054d-cbf5-4648-b3af-f6241b47a0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:31.662646 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939054d-cbf5-4648-b3af-f6241b47a0a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:31.663032 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939054d-cbf5-4648-b3af-f6241b47a0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:31.663306 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939054d-cbf5-4648-b3af-f6241b47a0a8 admin admin] PATH_INFO: `/v3/projects/26d47d43237f4b31bf2ce9e116b348f4/users/b83b9f5d4e2b455698fbf6ecee70cbda/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:31.665174 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6939054d-cbf5-4648-b3af-f6241b47a0a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26d47d43237f4b31bf2ce9e116b348f4, user_id=b83b9f5d4e2b455698fbf6ecee70cbda, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:31.667816 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6939054d-cbf5-4648-b3af-f6241b47a0a8 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:31.678213 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1249/2506] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:20:31 2022] PUT /identity/v3/projects/26d47d43237f4b31bf2ce9e116b348f4/users/b83b9f5d4e2b455698fbf6ecee70cbda/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:20:31.690264 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd047cf-afad-49b4-856a-81bc5024030b None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:31.690749 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd047cf-afad-49b4-856a-81bc5024030b None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:31.690876 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd047cf-afad-49b4-856a-81bc5024030b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:31.718585 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-6dd047cf-afad-49b4-856a-81bc5024030b None None] MFA Rules not processed for user `b83b9f5d4e2b455698fbf6ecee70cbda`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:20:31.719833 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-6dd047cf-afad-49b4-856a-81bc5024030b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:31.784729 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1258/2507] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:20:31 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:20:31.796567 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6939054d-cbf5-4648-b3af-f6241b47a0a8 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:31.851923 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6939054d-cbf5-4648-b3af-f6241b47a0a8 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:31.855524 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b375f4-fbae-4699-aa99-cda61fa74e2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:31.856121 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b375f4-fbae-4699-aa99-cda61fa74e2f service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:31.856398 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b375f4-fbae-4699-aa99-cda61fa74e2f service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:31.856674 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b375f4-fbae-4699-aa99-cda61fa74e2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:31.865485 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b375f4-fbae-4699-aa99-cda61fa74e2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:31.866930 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b375f4-fbae-4699-aa99-cda61fa74e2f service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:31.876746 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1250/2508] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:20:31 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:20:31.952373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd047cf-afad-49b4-856a-81bc5024030b None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:31.961911 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd047cf-afad-49b4-856a-81bc5024030b None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:31.965503 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc745413-3345-42e3-a114-4869d4a1e199 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:31.966093 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc745413-3345-42e3-a114-4869d4a1e199 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:31.966374 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc745413-3345-42e3-a114-4869d4a1e199 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:31.966646 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc745413-3345-42e3-a114-4869d4a1e199 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:32.021987 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc745413-3345-42e3-a114-4869d4a1e199 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:32.023418 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc745413-3345-42e3-a114-4869d4a1e199 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:32.032496 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1259/2509] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:20:31 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:20:35.457462 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b375f4-fbae-4699-aa99-cda61fa74e2f service cinder] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:35.466871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b375f4-fbae-4699-aa99-cda61fa74e2f service cinder] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:35.470419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c453283-7a74-4370-bd04-2e74a2f4aceb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:35.471083 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c453283-7a74-4370-bd04-2e74a2f4aceb service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:35.471362 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c453283-7a74-4370-bd04-2e74a2f4aceb service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:35.471632 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c453283-7a74-4370-bd04-2e74a2f4aceb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:35.534808 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c453283-7a74-4370-bd04-2e74a2f4aceb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:35.536282 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c453283-7a74-4370-bd04-2e74a2f4aceb service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:35.545795 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1251/2510] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:20:35 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:20:42.028117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=66 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:20:43.995469 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc745413-3345-42e3-a114-4869d4a1e199 service cinder] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:44.055478 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc745413-3345-42e3-a114-4869d4a1e199 service cinder] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:44.059939 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7eb2f-0553-421a-b6e9-40dbf5a69eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:44.061301 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7eb2f-0553-421a-b6e9-40dbf5a69eba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:44.061608 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7eb2f-0553-421a-b6e9-40dbf5a69eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:44.061882 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7eb2f-0553-421a-b6e9-40dbf5a69eba admin admin] PATH_INFO: `/v3/users/e62c30cf3c6244a6a750f0339eccaf5f` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:44.063025 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f7eb2f-0553-421a-b6e9-40dbf5a69eba admin admin] RBAC: Authorizing `identity:delete_user(user_id=e62c30cf3c6244a6a750f0339eccaf5f)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:44.064898 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f7eb2f-0553-421a-b6e9-40dbf5a69eba admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:44.109199 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f4f7eb2f-0553-421a-b6e9-40dbf5a69eba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e62c30cf3c6244a6a750f0339eccaf5f'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:44.116267 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f4f7eb2f-0553-421a-b6e9-40dbf5a69eba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e62c30cf3c6244a6a750f0339eccaf5f'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:44.116950 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f4f7eb2f-0553-421a-b6e9-40dbf5a69eba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e62c30cf3c6244a6a750f0339eccaf5f'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:44.117539 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f4f7eb2f-0553-421a-b6e9-40dbf5a69eba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e62c30cf3c6244a6a750f0339eccaf5f'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:44.125131 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f4f7eb2f-0553-421a-b6e9-40dbf5a69eba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e62c30cf3c6244a6a750f0339eccaf5f'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:44.141935 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1260/2511] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:20:43 2022] DELETE /identity/v3/users/e62c30cf3c6244a6a750f0339eccaf5f => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:20:44.146449 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4328608-b965-4365-a63a-d7c7f78f3b39 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:44.146951 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4328608-b965-4365-a63a-d7c7f78f3b39 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:44.147052 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4328608-b965-4365-a63a-d7c7f78f3b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:44.165356 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-d4328608-b965-4365-a63a-d7c7f78f3b39 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:20:44.166535 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-d4328608-b965-4365-a63a-d7c7f78f3b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:44.184574 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:20:44.225025 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1252/2512] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:20:44.235453 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7eb2f-0553-421a-b6e9-40dbf5a69eba admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:44.236874 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f4f7eb2f-0553-421a-b6e9-40dbf5a69eba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:44.291368 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7eb2f-0553-421a-b6e9-40dbf5a69eba admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:44.294952 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0026f4-4f4d-44c8-87c9-bb61572cc98c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:44.296045 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0026f4-4f4d-44c8-87c9-bb61572cc98c service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:44.296325 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0026f4-4f4d-44c8-87c9-bb61572cc98c service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:44.296596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0026f4-4f4d-44c8-87c9-bb61572cc98c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:44.305377 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0026f4-4f4d-44c8-87c9-bb61572cc98c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:44.306765 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0026f4-4f4d-44c8-87c9-bb61572cc98c service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:44.317272 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1261/2513] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:20:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:20:44.590789 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4328608-b965-4365-a63a-d7c7f78f3b39 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:44.592758 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-d4328608-b965-4365-a63a-d7c7f78f3b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:44.608400 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4328608-b965-4365-a63a-d7c7f78f3b39 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:44.614031 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc295faa-79f2-41ce-97dc-5564c817b8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:44.614705 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc295faa-79f2-41ce-97dc-5564c817b8e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:44.614970 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc295faa-79f2-41ce-97dc-5564c817b8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:44.615242 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc295faa-79f2-41ce-97dc-5564c817b8e2 admin admin] PATH_INFO: `/v3/projects/a0f854317e924e6b8317a35f1e3d038f` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:44.616357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc295faa-79f2-41ce-97dc-5564c817b8e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0f854317e924e6b8317a35f1e3d038f)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:44.618248 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc295faa-79f2-41ce-97dc-5564c817b8e2 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:44.665055 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-bc295faa-79f2-41ce-97dc-5564c817b8e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0f854317e924e6b8317a35f1e3d038f', 'request_id': 'req-bc295faa-79f2-41ce-97dc-5564c817b8e2'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:44.671034 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-bc295faa-79f2-41ce-97dc-5564c817b8e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0f854317e924e6b8317a35f1e3d038f', 'request_id': 'req-bc295faa-79f2-41ce-97dc-5564c817b8e2'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:44.677874 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-bc295faa-79f2-41ce-97dc-5564c817b8e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0f854317e924e6b8317a35f1e3d038f', 'request_id': 'req-bc295faa-79f2-41ce-97dc-5564c817b8e2'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:44.689401 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-bc295faa-79f2-41ce-97dc-5564c817b8e2 admin admin] The token cache is being invalidate because project a0f854317e924e6b8317a35f1e3d038f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:20:44.689767 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-bc295faa-79f2-41ce-97dc-5564c817b8e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:44.690466 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-bc295faa-79f2-41ce-97dc-5564c817b8e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:44.692139 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1253/2514] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:20:44 2022] DELETE /identity/v3/projects/a0f854317e924e6b8317a35f1e3d038f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:20:44.696950 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0026f4-4f4d-44c8-87c9-bb61572cc98c service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:44.706695 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0026f4-4f4d-44c8-87c9-bb61572cc98c service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:44.724643 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:20:44.769180 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc5d6f-c0b7-4019-8d24-93fe6e17e6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:44.769812 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc5d6f-c0b7-4019-8d24-93fe6e17e6e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:44.770105 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc5d6f-c0b7-4019-8d24-93fe6e17e6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:44.770347 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc5d6f-c0b7-4019-8d24-93fe6e17e6e4 admin admin] PATH_INFO: `/v3/users/b83b9f5d4e2b455698fbf6ecee70cbda` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:44.771482 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc5d6f-c0b7-4019-8d24-93fe6e17e6e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b83b9f5d4e2b455698fbf6ecee70cbda)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:44.773411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc5d6f-c0b7-4019-8d24-93fe6e17e6e4 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:44.818148 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-9dfc5d6f-c0b7-4019-8d24-93fe6e17e6e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b83b9f5d4e2b455698fbf6ecee70cbda'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:44.825992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-9dfc5d6f-c0b7-4019-8d24-93fe6e17e6e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b83b9f5d4e2b455698fbf6ecee70cbda'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:44.826752 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-9dfc5d6f-c0b7-4019-8d24-93fe6e17e6e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b83b9f5d4e2b455698fbf6ecee70cbda'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:44.827384 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-9dfc5d6f-c0b7-4019-8d24-93fe6e17e6e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b83b9f5d4e2b455698fbf6ecee70cbda'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:44.834509 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-9dfc5d6f-c0b7-4019-8d24-93fe6e17e6e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b83b9f5d4e2b455698fbf6ecee70cbda'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:44.851834 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1262/2515] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:20:44 2022] DELETE /identity/v3/users/b83b9f5d4e2b455698fbf6ecee70cbda => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:20:45.118237 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc295faa-79f2-41ce-97dc-5564c817b8e2 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:45.119706 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-bc295faa-79f2-41ce-97dc-5564c817b8e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:45.132684 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc295faa-79f2-41ce-97dc-5564c817b8e2 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:45.189190 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be664d-93b6-44bb-ab48-6e164623dd72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:45.189827 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be664d-93b6-44bb-ab48-6e164623dd72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:45.190069 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be664d-93b6-44bb-ab48-6e164623dd72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:45.190337 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be664d-93b6-44bb-ab48-6e164623dd72 admin admin] PATH_INFO: `/v3/projects/26d47d43237f4b31bf2ce9e116b348f4` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:45.191437 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1be664d-93b6-44bb-ab48-6e164623dd72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26d47d43237f4b31bf2ce9e116b348f4)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:45.193311 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1be664d-93b6-44bb-ab48-6e164623dd72 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:45.238752 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-a1be664d-93b6-44bb-ab48-6e164623dd72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26d47d43237f4b31bf2ce9e116b348f4', 'request_id': 'req-a1be664d-93b6-44bb-ab48-6e164623dd72'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:45.239843 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:20:45.245460 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-a1be664d-93b6-44bb-ab48-6e164623dd72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26d47d43237f4b31bf2ce9e116b348f4', 'request_id': 'req-a1be664d-93b6-44bb-ab48-6e164623dd72'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:45.254072 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-a1be664d-93b6-44bb-ab48-6e164623dd72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26d47d43237f4b31bf2ce9e116b348f4', 'request_id': 'req-a1be664d-93b6-44bb-ab48-6e164623dd72'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:45.265831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-a1be664d-93b6-44bb-ab48-6e164623dd72 admin admin] The token cache is being invalidate because project 26d47d43237f4b31bf2ce9e116b348f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:20:45.266120 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-a1be664d-93b6-44bb-ab48-6e164623dd72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:45.266743 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-a1be664d-93b6-44bb-ab48-6e164623dd72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:45.268233 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1254/2516] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:20:45 2022] DELETE /identity/v3/projects/26d47d43237f4b31bf2ce9e116b348f4 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:20:45.273213 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a8cf0-09a8-46b5-9cbe-483718857535 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:45.273687 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a8cf0-09a8-46b5-9cbe-483718857535 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:45.273884 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a8cf0-09a8-46b5-9cbe-483718857535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:45.293112 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-215a8cf0-09a8-46b5-9cbe-483718857535 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:20:45.294351 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-215a8cf0-09a8-46b5-9cbe-483718857535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:45.351383 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1263/2517] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:20:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:20:45.359122 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022bef2-bc1c-4164-996c-e745f6681724 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:45.359409 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022bef2-bc1c-4164-996c-e745f6681724 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:45.359685 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022bef2-bc1c-4164-996c-e745f6681724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:45.379428 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-3022bef2-bc1c-4164-996c-e745f6681724 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:20:45.381066 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-3022bef2-bc1c-4164-996c-e745f6681724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:45.389488 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1255/2518] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:20:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:20:45.393472 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a8cf0-09a8-46b5-9cbe-483718857535 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:45.403165 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a8cf0-09a8-46b5-9cbe-483718857535 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:45.407870 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0953eaca-812a-4d90-8abd-d7a97180ac97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:45.408461 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0953eaca-812a-4d90-8abd-d7a97180ac97 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:45.408739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0953eaca-812a-4d90-8abd-d7a97180ac97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:45.409007 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0953eaca-812a-4d90-8abd-d7a97180ac97 admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:45.409641 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0953eaca-812a-4d90-8abd-d7a97180ac97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:20:45.409943 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0953eaca-812a-4d90-8abd-d7a97180ac97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:45.411557 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0953eaca-812a-4d90-8abd-d7a97180ac97 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:45.422891 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1264/2519] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:20:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:20:45.427091 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022bef2-bc1c-4164-996c-e745f6681724 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:45.437121 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022bef2-bc1c-4164-996c-e745f6681724 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:45.441756 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f916e33-8394-4637-936e-0e5aa3749997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:45.442340 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f916e33-8394-4637-936e-0e5aa3749997 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:45.442636 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f916e33-8394-4637-936e-0e5aa3749997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:45.442930 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f916e33-8394-4637-936e-0e5aa3749997 admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:45.443556 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f916e33-8394-4637-936e-0e5aa3749997 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:45.445421 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f916e33-8394-4637-936e-0e5aa3749997 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:45.473018 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1256/2520] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:20:45 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:20:45.477040 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0953eaca-812a-4d90-8abd-d7a97180ac97 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:45.487166 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0953eaca-812a-4d90-8abd-d7a97180ac97 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:45.513682 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:20:45.545703 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab557efd-b11b-4dd2-a99e-a9d9af8423ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:45.545852 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab557efd-b11b-4dd2-a99e-a9d9af8423ad admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:45.546151 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab557efd-b11b-4dd2-a99e-a9d9af8423ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:45.546400 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab557efd-b11b-4dd2-a99e-a9d9af8423ad admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:45.547058 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab557efd-b11b-4dd2-a99e-a9d9af8423ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:45.548937 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab557efd-b11b-4dd2-a99e-a9d9af8423ad admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:45.572459 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1265/2521] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:20:45 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:20:45.577189 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f916e33-8394-4637-936e-0e5aa3749997 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:45.587486 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f916e33-8394-4637-936e-0e5aa3749997 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:45.592069 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35073d83-60ca-4b58-bdee-a6b4d784b026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:45.592656 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35073d83-60ca-4b58-bdee-a6b4d784b026 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:45.592934 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35073d83-60ca-4b58-bdee-a6b4d784b026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:45.593203 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35073d83-60ca-4b58-bdee-a6b4d784b026 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:45.593868 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35073d83-60ca-4b58-bdee-a6b4d784b026 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:45.595521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35073d83-60ca-4b58-bdee-a6b4d784b026 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:45.605217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1257/2522] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:20:45.608948 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab557efd-b11b-4dd2-a99e-a9d9af8423ad admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:45.618460 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab557efd-b11b-4dd2-a99e-a9d9af8423ad admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:45.624494 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa318a4-a771-4814-893c-e634151822f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:45.625307 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa318a4-a771-4814-893c-e634151822f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:45.625622 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa318a4-a771-4814-893c-e634151822f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:45.625994 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa318a4-a771-4814-893c-e634151822f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:45.626630 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa318a4-a771-4814-893c-e634151822f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:45.628217 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa318a4-a771-4814-893c-e634151822f5 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:45.638458 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1266/2523] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:20:45.643149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35073d83-60ca-4b58-bdee-a6b4d784b026 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:45.653015 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35073d83-60ca-4b58-bdee-a6b4d784b026 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:45.657612 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b4113-3b2c-4752-a1a2-8a0e8f4ff652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:45.658243 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b4113-3b2c-4752-a1a2-8a0e8f4ff652 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:45.658521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b4113-3b2c-4752-a1a2-8a0e8f4ff652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:45.658819 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b4113-3b2c-4752-a1a2-8a0e8f4ff652 admin admin] PATH_INFO: `/v3/projects/af220e0a9621477f9defcaf687ec0e75/users/23303c3ccb8948d0a15648ebf518d4f8/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:45.673478 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4b4113-3b2c-4752-a1a2-8a0e8f4ff652 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af220e0a9621477f9defcaf687ec0e75, user_id=23303c3ccb8948d0a15648ebf518d4f8, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:45.676139 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4b4113-3b2c-4752-a1a2-8a0e8f4ff652 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:45.689330 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1258/2524] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:20:45 2022] PUT /identity/v3/projects/af220e0a9621477f9defcaf687ec0e75/users/23303c3ccb8948d0a15648ebf518d4f8/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:20:45.700651 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075b0b9-7df7-40d8-8280-b827c352d284 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:45.700929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075b0b9-7df7-40d8-8280-b827c352d284 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:45.701198 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075b0b9-7df7-40d8-8280-b827c352d284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:45.728595 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-8075b0b9-7df7-40d8-8280-b827c352d284 None None] MFA Rules not processed for user `23303c3ccb8948d0a15648ebf518d4f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:20:45.729802 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-8075b0b9-7df7-40d8-8280-b827c352d284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:45.792016 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1267/2525] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:20:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:20:45.796963 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b4113-3b2c-4752-a1a2-8a0e8f4ff652 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:45.858794 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b4113-3b2c-4752-a1a2-8a0e8f4ff652 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:45.863413 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a106400-ab81-4027-8201-1cb0d8466498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:45.864010 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a106400-ab81-4027-8201-1cb0d8466498 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:45.864290 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a106400-ab81-4027-8201-1cb0d8466498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:45.864563 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a106400-ab81-4027-8201-1cb0d8466498 admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:45.865186 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a106400-ab81-4027-8201-1cb0d8466498 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:45.867101 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a106400-ab81-4027-8201-1cb0d8466498 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:45.893702 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1259/2526] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:20:45 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:20:45.897876 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075b0b9-7df7-40d8-8280-b827c352d284 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:45.909055 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075b0b9-7df7-40d8-8280-b827c352d284 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:45.964819 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46dbc6-1202-4e47-8515-bf41df3c0a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:45.965422 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46dbc6-1202-4e47-8515-bf41df3c0a8f admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:45.965700 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46dbc6-1202-4e47-8515-bf41df3c0a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:45.965970 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46dbc6-1202-4e47-8515-bf41df3c0a8f admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:45.966593 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e46dbc6-1202-4e47-8515-bf41df3c0a8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:45.969087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e46dbc6-1202-4e47-8515-bf41df3c0a8f admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:45.989361 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1268/2527] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:20:45 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:20:45.993468 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a106400-ab81-4027-8201-1cb0d8466498 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:46.004383 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a106400-ab81-4027-8201-1cb0d8466498 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:46.009030 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb06950d-8a3e-443b-bea7-c1455b93243e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:46.009619 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06950d-8a3e-443b-bea7-c1455b93243e admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:46.009896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06950d-8a3e-443b-bea7-c1455b93243e admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:46.010167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06950d-8a3e-443b-bea7-c1455b93243e admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:46.010802 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb06950d-8a3e-443b-bea7-c1455b93243e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:46.012427 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb06950d-8a3e-443b-bea7-c1455b93243e admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:46.023257 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1260/2528] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:20:46.027183 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46dbc6-1202-4e47-8515-bf41df3c0a8f admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:46.036849 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46dbc6-1202-4e47-8515-bf41df3c0a8f admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:46.041432 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e9442f-3561-4cf1-a26e-a6eca73182b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:46.042113 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9442f-3561-4cf1-a26e-a6eca73182b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:46.042343 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9442f-3561-4cf1-a26e-a6eca73182b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:46.042613 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9442f-3561-4cf1-a26e-a6eca73182b2 admin admin] PATH_INFO: `/v3/projects/2dcbf7b69d924f35a5514122cd64527c/users/46a295124b8f4204b51849607078862e/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:46.056852 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e9442f-3561-4cf1-a26e-a6eca73182b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dcbf7b69d924f35a5514122cd64527c, user_id=46a295124b8f4204b51849607078862e, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:46.059474 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e9442f-3561-4cf1-a26e-a6eca73182b2 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:46.073017 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1269/2529] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:20:46 2022] PUT /identity/v3/projects/2dcbf7b69d924f35a5514122cd64527c/users/46a295124b8f4204b51849607078862e/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:20:46.077503 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb06950d-8a3e-443b-bea7-c1455b93243e admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:46.087623 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb06950d-8a3e-443b-bea7-c1455b93243e admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:46.130852 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:20:46.144360 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364815c-fd8c-48f1-a53e-615b4e0f580a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:46.144962 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364815c-fd8c-48f1-a53e-615b4e0f580a admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:46.145349 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364815c-fd8c-48f1-a53e-615b4e0f580a admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:46.145624 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364815c-fd8c-48f1-a53e-615b4e0f580a admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:46.146262 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3364815c-fd8c-48f1-a53e-615b4e0f580a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:46.147906 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3364815c-fd8c-48f1-a53e-615b4e0f580a admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:46.157528 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1261/2530] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:20:46.161615 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e9442f-3561-4cf1-a26e-a6eca73182b2 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:46.167093 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:20:46.173385 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e9442f-3561-4cf1-a26e-a6eca73182b2 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:46.178221 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c2065a-904b-4d94-82fe-12f95d73b6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:46.178897 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2065a-904b-4d94-82fe-12f95d73b6fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:46.179178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2065a-904b-4d94-82fe-12f95d73b6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:46.179450 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2065a-904b-4d94-82fe-12f95d73b6fa admin admin] PATH_INFO: `/v3/projects/2dcbf7b69d924f35a5514122cd64527c/users/46a295124b8f4204b51849607078862e/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:46.181370 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c2065a-904b-4d94-82fe-12f95d73b6fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dcbf7b69d924f35a5514122cd64527c, user_id=46a295124b8f4204b51849607078862e, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:46.184073 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c2065a-904b-4d94-82fe-12f95d73b6fa admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:46.194568 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1270/2531] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:20:46 2022] PUT /identity/v3/projects/2dcbf7b69d924f35a5514122cd64527c/users/46a295124b8f4204b51849607078862e/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:20:46.205140 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b189bc-4000-44ff-9a15-6f1fac5ed458 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:46.205634 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b189bc-4000-44ff-9a15-6f1fac5ed458 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:46.205731 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b189bc-4000-44ff-9a15-6f1fac5ed458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:46.233199 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-67b189bc-4000-44ff-9a15-6f1fac5ed458 None None] MFA Rules not processed for user `46a295124b8f4204b51849607078862e`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:20:46.234412 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-67b189bc-4000-44ff-9a15-6f1fac5ed458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:46.301293 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1262/2532] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:20:46 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:20:46.313612 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c2065a-904b-4d94-82fe-12f95d73b6fa admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:46.315428 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-46c2065a-904b-4d94-82fe-12f95d73b6fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:46.370508 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c2065a-904b-4d94-82fe-12f95d73b6fa admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:46.374631 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c701bd-ae41-4c83-80ec-e654726cba54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:46.375254 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c701bd-ae41-4c83-80ec-e654726cba54 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:46.375531 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c701bd-ae41-4c83-80ec-e654726cba54 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:46.375801 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c701bd-ae41-4c83-80ec-e654726cba54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:46.384509 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c701bd-ae41-4c83-80ec-e654726cba54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:46.385904 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c701bd-ae41-4c83-80ec-e654726cba54 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:46.395605 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1271/2533] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:20:46 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:20:46.483997 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b189bc-4000-44ff-9a15-6f1fac5ed458 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:46.493624 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b189bc-4000-44ff-9a15-6f1fac5ed458 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:46.497543 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ea23d-7479-4a9e-b843-93c1b46db0ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:46.498888 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ea23d-7479-4a9e-b843-93c1b46db0ec service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:46.499080 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ea23d-7479-4a9e-b843-93c1b46db0ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:46.499356 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ea23d-7479-4a9e-b843-93c1b46db0ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:46.554318 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9ea23d-7479-4a9e-b843-93c1b46db0ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:46.555726 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9ea23d-7479-4a9e-b843-93c1b46db0ec service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:46.566028 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1263/2534] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:20:46 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:20:46.829634 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c701bd-ae41-4c83-80ec-e654726cba54 service cinder] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:46.890147 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c701bd-ae41-4c83-80ec-e654726cba54 service cinder] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:46.894515 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd33c0-ea29-41fd-969f-14543861dbaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:46.895227 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd33c0-ea29-41fd-969f-14543861dbaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:46.895503 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd33c0-ea29-41fd-969f-14543861dbaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:46.895769 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd33c0-ea29-41fd-969f-14543861dbaf admin admin] PATH_INFO: `/v3/users/23303c3ccb8948d0a15648ebf518d4f8` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:46.896859 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bd33c0-ea29-41fd-969f-14543861dbaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=23303c3ccb8948d0a15648ebf518d4f8)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:46.898732 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bd33c0-ea29-41fd-969f-14543861dbaf admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:46.944152 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d7bd33c0-ea29-41fd-969f-14543861dbaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23303c3ccb8948d0a15648ebf518d4f8'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:46.950766 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d7bd33c0-ea29-41fd-969f-14543861dbaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23303c3ccb8948d0a15648ebf518d4f8'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:46.951403 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d7bd33c0-ea29-41fd-969f-14543861dbaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23303c3ccb8948d0a15648ebf518d4f8'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:46.952006 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d7bd33c0-ea29-41fd-969f-14543861dbaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23303c3ccb8948d0a15648ebf518d4f8'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:46.958853 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d7bd33c0-ea29-41fd-969f-14543861dbaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23303c3ccb8948d0a15648ebf518d4f8'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:46.976066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1272/2535] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:20:46 2022] DELETE /identity/v3/users/23303c3ccb8948d0a15648ebf518d4f8 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:20:46.984440 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02026848-aa59-4ef5-a5da-9e76015f6290 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:46.984731 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02026848-aa59-4ef5-a5da-9e76015f6290 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:46.985007 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02026848-aa59-4ef5-a5da-9e76015f6290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:47.003663 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-02026848-aa59-4ef5-a5da-9e76015f6290 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:20:47.004855 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-02026848-aa59-4ef5-a5da-9e76015f6290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:47.065664 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1264/2536] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:20:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:20:47.075410 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd33c0-ea29-41fd-969f-14543861dbaf admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:47.076810 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d7bd33c0-ea29-41fd-969f-14543861dbaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:47.090060 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:20:47.137256 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd33c0-ea29-41fd-969f-14543861dbaf admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:47.141423 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31dc32-a730-4416-b1a4-43e50cc148ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:47.142008 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31dc32-a730-4416-b1a4-43e50cc148ae service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:47.142289 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31dc32-a730-4416-b1a4-43e50cc148ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:47.142562 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31dc32-a730-4416-b1a4-43e50cc148ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:47.151679 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a31dc32-a730-4416-b1a4-43e50cc148ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:47.153082 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a31dc32-a730-4416-b1a4-43e50cc148ae service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:47.163363 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1273/2537] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:20:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:20:47.418670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02026848-aa59-4ef5-a5da-9e76015f6290 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:47.420265 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-02026848-aa59-4ef5-a5da-9e76015f6290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:47.432698 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02026848-aa59-4ef5-a5da-9e76015f6290 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:47.437833 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094af466-78c3-43d3-8404-1ac9ac1ebb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:47.438468 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094af466-78c3-43d3-8404-1ac9ac1ebb78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:47.438749 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094af466-78c3-43d3-8404-1ac9ac1ebb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:47.439105 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094af466-78c3-43d3-8404-1ac9ac1ebb78 admin admin] PATH_INFO: `/v3/projects/af220e0a9621477f9defcaf687ec0e75` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:47.440231 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094af466-78c3-43d3-8404-1ac9ac1ebb78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af220e0a9621477f9defcaf687ec0e75)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:47.442270 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094af466-78c3-43d3-8404-1ac9ac1ebb78 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:47.445144 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:20:47.489795 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-094af466-78c3-43d3-8404-1ac9ac1ebb78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af220e0a9621477f9defcaf687ec0e75', 'request_id': 'req-094af466-78c3-43d3-8404-1ac9ac1ebb78'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:47.495384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-094af466-78c3-43d3-8404-1ac9ac1ebb78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af220e0a9621477f9defcaf687ec0e75', 'request_id': 'req-094af466-78c3-43d3-8404-1ac9ac1ebb78'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:47.503779 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-094af466-78c3-43d3-8404-1ac9ac1ebb78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af220e0a9621477f9defcaf687ec0e75', 'request_id': 'req-094af466-78c3-43d3-8404-1ac9ac1ebb78'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:47.515330 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-094af466-78c3-43d3-8404-1ac9ac1ebb78 admin admin] The token cache is being invalidate because project af220e0a9621477f9defcaf687ec0e75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:20:47.515619 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-094af466-78c3-43d3-8404-1ac9ac1ebb78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:47.516244 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-094af466-78c3-43d3-8404-1ac9ac1ebb78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:47.517725 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1265/2538] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:20:47 2022] DELETE /identity/v3/projects/af220e0a9621477f9defcaf687ec0e75 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:20:47.521696 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31dc32-a730-4416-b1a4-43e50cc148ae service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:47.531342 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31dc32-a730-4416-b1a4-43e50cc148ae service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:47.585205 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2140e324-c693-4598-86bf-9224ff24dfe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:47.585820 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140e324-c693-4598-86bf-9224ff24dfe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:47.586105 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140e324-c693-4598-86bf-9224ff24dfe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:47.586377 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140e324-c693-4598-86bf-9224ff24dfe8 admin admin] PATH_INFO: `/v3/users/46a295124b8f4204b51849607078862e` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:47.587496 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2140e324-c693-4598-86bf-9224ff24dfe8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46a295124b8f4204b51849607078862e)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:47.589376 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2140e324-c693-4598-86bf-9224ff24dfe8 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:47.643501 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2140e324-c693-4598-86bf-9224ff24dfe8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46a295124b8f4204b51849607078862e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:47.651215 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2140e324-c693-4598-86bf-9224ff24dfe8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46a295124b8f4204b51849607078862e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:47.651837 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2140e324-c693-4598-86bf-9224ff24dfe8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46a295124b8f4204b51849607078862e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:47.652438 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2140e324-c693-4598-86bf-9224ff24dfe8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46a295124b8f4204b51849607078862e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:47.659338 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2140e324-c693-4598-86bf-9224ff24dfe8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46a295124b8f4204b51849607078862e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:47.677374 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1274/2539] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:20:47 2022] DELETE /identity/v3/users/46a295124b8f4204b51849607078862e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:20:47.957956 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094af466-78c3-43d3-8404-1ac9ac1ebb78 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:47.959456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-094af466-78c3-43d3-8404-1ac9ac1ebb78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:47.972666 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094af466-78c3-43d3-8404-1ac9ac1ebb78 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:48.031018 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54fa387-2580-4a0e-bcec-f98c12a0923d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:48.031650 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54fa387-2580-4a0e-bcec-f98c12a0923d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:48.031927 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54fa387-2580-4a0e-bcec-f98c12a0923d admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:48.032198 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54fa387-2580-4a0e-bcec-f98c12a0923d admin admin] PATH_INFO: `/v3/projects/2dcbf7b69d924f35a5514122cd64527c` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:48.033293 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54fa387-2580-4a0e-bcec-f98c12a0923d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dcbf7b69d924f35a5514122cd64527c)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:48.035237 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54fa387-2580-4a0e-bcec-f98c12a0923d admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:48.080833 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-b54fa387-2580-4a0e-bcec-f98c12a0923d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dcbf7b69d924f35a5514122cd64527c', 'request_id': 'req-b54fa387-2580-4a0e-bcec-f98c12a0923d'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:48.086431 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-b54fa387-2580-4a0e-bcec-f98c12a0923d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dcbf7b69d924f35a5514122cd64527c', 'request_id': 'req-b54fa387-2580-4a0e-bcec-f98c12a0923d'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:48.093521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-b54fa387-2580-4a0e-bcec-f98c12a0923d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dcbf7b69d924f35a5514122cd64527c', 'request_id': 'req-b54fa387-2580-4a0e-bcec-f98c12a0923d'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:48.104413 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-b54fa387-2580-4a0e-bcec-f98c12a0923d admin admin] The token cache is being invalidate because project 2dcbf7b69d924f35a5514122cd64527c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:20:48.104704 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-b54fa387-2580-4a0e-bcec-f98c12a0923d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:48.105326 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-b54fa387-2580-4a0e-bcec-f98c12a0923d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:20:48.106775 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1266/2540] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:20:47 2022] DELETE /identity/v3/projects/2dcbf7b69d924f35a5514122cd64527c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:20:48.112180 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659853ff-0eca-49fe-9a59-5c55ed2e7268 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:48.112460 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659853ff-0eca-49fe-9a59-5c55ed2e7268 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:48.112726 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659853ff-0eca-49fe-9a59-5c55ed2e7268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:48.132285 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-659853ff-0eca-49fe-9a59-5c55ed2e7268 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:20:48.133391 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-659853ff-0eca-49fe-9a59-5c55ed2e7268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:48.160774 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:20:48.192126 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1275/2541] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:20:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:20:48.200066 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7736786-2230-4a55-a92e-7c2428d0cb1c None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:48.200353 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7736786-2230-4a55-a92e-7c2428d0cb1c None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:48.200631 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7736786-2230-4a55-a92e-7c2428d0cb1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:48.220007 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-e7736786-2230-4a55-a92e-7c2428d0cb1c None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:20:48.221235 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-e7736786-2230-4a55-a92e-7c2428d0cb1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:48.229097 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1267/2542] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:20:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:20:48.233649 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659853ff-0eca-49fe-9a59-5c55ed2e7268 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:48.243127 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659853ff-0eca-49fe-9a59-5c55ed2e7268 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:48.247552 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f649408e-def0-4959-8b08-5238e838bbb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:48.248736 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649408e-def0-4959-8b08-5238e838bbb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:48.249038 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649408e-def0-4959-8b08-5238e838bbb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:48.249305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649408e-def0-4959-8b08-5238e838bbb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:48.249944 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f649408e-def0-4959-8b08-5238e838bbb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:20:48.250239 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f649408e-def0-4959-8b08-5238e838bbb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:48.251831 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f649408e-def0-4959-8b08-5238e838bbb2 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:48.263078 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1276/2543] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:20:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:20:48.267096 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7736786-2230-4a55-a92e-7c2428d0cb1c None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:48.276116 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7736786-2230-4a55-a92e-7c2428d0cb1c None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:48.281174 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042eccf1-22f2-412c-aceb-7887b95a6fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:48.281757 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042eccf1-22f2-412c-aceb-7887b95a6fc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:48.282035 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042eccf1-22f2-412c-aceb-7887b95a6fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:48.282321 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042eccf1-22f2-412c-aceb-7887b95a6fc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:48.282975 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042eccf1-22f2-412c-aceb-7887b95a6fc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:48.284842 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042eccf1-22f2-412c-aceb-7887b95a6fc7 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:48.299763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:20:48.314116 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1268/2544] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:20:48 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:20:48.318214 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f649408e-def0-4959-8b08-5238e838bbb2 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:48.327992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f649408e-def0-4959-8b08-5238e838bbb2 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:48.382904 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec4b7d6-e1d6-45d3-a25b-5e0956a37a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:48.383432 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec4b7d6-e1d6-45d3-a25b-5e0956a37a16 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:48.383705 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec4b7d6-e1d6-45d3-a25b-5e0956a37a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:48.384035 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec4b7d6-e1d6-45d3-a25b-5e0956a37a16 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:48.384657 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec4b7d6-e1d6-45d3-a25b-5e0956a37a16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:48.386549 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec4b7d6-e1d6-45d3-a25b-5e0956a37a16 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:48.414691 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1277/2545] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:20:48 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:20:48.418883 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042eccf1-22f2-412c-aceb-7887b95a6fc7 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:48.429776 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042eccf1-22f2-412c-aceb-7887b95a6fc7 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:48.434303 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44cb38-5a9c-4ef2-9d66-178c80f77a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:48.434917 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44cb38-5a9c-4ef2-9d66-178c80f77a4f admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:48.435193 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44cb38-5a9c-4ef2-9d66-178c80f77a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:48.435737 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44cb38-5a9c-4ef2-9d66-178c80f77a4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:48.436380 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b44cb38-5a9c-4ef2-9d66-178c80f77a4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:48.437987 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b44cb38-5a9c-4ef2-9d66-178c80f77a4f admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:48.447712 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1269/2546] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:20:48.451671 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec4b7d6-e1d6-45d3-a25b-5e0956a37a16 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:48.461602 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec4b7d6-e1d6-45d3-a25b-5e0956a37a16 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:48.466259 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0df7a-ac58-49d2-845a-ac7c7a41e8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:48.467387 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0df7a-ac58-49d2-845a-ac7c7a41e8c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:48.467667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0df7a-ac58-49d2-845a-ac7c7a41e8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:48.467940 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0df7a-ac58-49d2-845a-ac7c7a41e8c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:48.468576 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf0df7a-ac58-49d2-845a-ac7c7a41e8c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:48.470172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf0df7a-ac58-49d2-845a-ac7c7a41e8c8 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:48.481631 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1278/2547] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:20:48.486294 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44cb38-5a9c-4ef2-9d66-178c80f77a4f admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:48.495995 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44cb38-5a9c-4ef2-9d66-178c80f77a4f admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:48.500515 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22009d6-f7b8-49b1-99ba-7a7b426e58c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:48.501147 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22009d6-f7b8-49b1-99ba-7a7b426e58c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:48.501423 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22009d6-f7b8-49b1-99ba-7a7b426e58c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:48.501693 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22009d6-f7b8-49b1-99ba-7a7b426e58c8 admin admin] PATH_INFO: `/v3/projects/7a92484ba35b46b6bd86a3072403410e/users/4c26b531c07a498c9617c9ee2f7bf7e9/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:48.516036 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22009d6-f7b8-49b1-99ba-7a7b426e58c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a92484ba35b46b6bd86a3072403410e, user_id=4c26b531c07a498c9617c9ee2f7bf7e9, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:48.518708 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22009d6-f7b8-49b1-99ba-7a7b426e58c8 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:48.531766 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1270/2548] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:20:48 2022] PUT /identity/v3/projects/7a92484ba35b46b6bd86a3072403410e/users/4c26b531c07a498c9617c9ee2f7bf7e9/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:20:48.536764 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29c854-a7db-41b2-b67c-3a267752a419 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:48.537237 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29c854-a7db-41b2-b67c-3a267752a419 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:48.537366 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29c854-a7db-41b2-b67c-3a267752a419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:48.555985 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-5e29c854-a7db-41b2-b67c-3a267752a419 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:20:48.557155 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-5e29c854-a7db-41b2-b67c-3a267752a419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:48.614323 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1279/2549] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:20:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:20:48.624117 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22009d6-f7b8-49b1-99ba-7a7b426e58c8 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:48.625802 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-f22009d6-f7b8-49b1-99ba-7a7b426e58c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:48.681206 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22009d6-f7b8-49b1-99ba-7a7b426e58c8 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:48.684764 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9b113-e40f-4cd1-ac0a-3d357d1c7d04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:48.685654 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9b113-e40f-4cd1-ac0a-3d357d1c7d04 service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:48.685939 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9b113-e40f-4cd1-ac0a-3d357d1c7d04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:48.686211 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9b113-e40f-4cd1-ac0a-3d357d1c7d04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:48.695332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b9b113-e40f-4cd1-ac0a-3d357d1c7d04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:48.696746 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b9b113-e40f-4cd1-ac0a-3d357d1c7d04 service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:48.706606 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1271/2550] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:20:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:20:49.359384 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27fd0f-8603-45cb-b349-680c2bbd287d None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:49.359664 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27fd0f-8603-45cb-b349-680c2bbd287d None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:49.359932 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27fd0f-8603-45cb-b349-680c2bbd287d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:49.387772 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-cf27fd0f-8603-45cb-b349-680c2bbd287d None None] MFA Rules not processed for user `4c26b531c07a498c9617c9ee2f7bf7e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:20:49.389639 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-cf27fd0f-8603-45cb-b349-680c2bbd287d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:49.419036 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:20:49.456935 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1280/2551] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:20:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:20:49.468196 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9b113-e40f-4cd1-ac0a-3d357d1c7d04 service neutron] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:49.469709 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-a3b9b113-e40f-4cd1-ac0a-3d357d1c7d04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:49.527091 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9b113-e40f-4cd1-ac0a-3d357d1c7d04 service neutron] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:49.532489 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebfcc9b-92e7-405e-8853-77792d379323 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:49.533108 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfcc9b-92e7-405e-8853-77792d379323 service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:49.533453 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfcc9b-92e7-405e-8853-77792d379323 service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:49.533766 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfcc9b-92e7-405e-8853-77792d379323 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:49.542545 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebfcc9b-92e7-405e-8853-77792d379323 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:49.544015 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebfcc9b-92e7-405e-8853-77792d379323 service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:49.553409 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1272/2552] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:20:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:20:53.020037 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c9f63-ccbf-4439-9243-3759a6431afd None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:53.020550 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c9f63-ccbf-4439-9243-3759a6431afd None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:53.020652 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c9f63-ccbf-4439-9243-3759a6431afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:53.039008 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-c42c9f63-ccbf-4439-9243-3759a6431afd None None] MFA Rules not processed for user `4c26b531c07a498c9617c9ee2f7bf7e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:20:53.040203 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c42c9f63-ccbf-4439-9243-3759a6431afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:53.046940 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1281/2553] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:20:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:20:53.296306 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebfcc9b-92e7-405e-8853-77792d379323 service cinder] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:20:53.307158 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebfcc9b-92e7-405e-8853-77792d379323 service cinder] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:20:53.311685 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd6e02a-ca1d-4074-bb22-2b1d7110cb31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:20:53.312278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd6e02a-ca1d-4074-bb22-2b1d7110cb31 service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:20:53.312559 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd6e02a-ca1d-4074-bb22-2b1d7110cb31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:20:53.312822 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd6e02a-ca1d-4074-bb22-2b1d7110cb31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:20:53.314548 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-abd6e02a-ca1d-4074-bb22-2b1d7110cb31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:20:53.325483 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd6e02a-ca1d-4074-bb22-2b1d7110cb31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:20:53.327700 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd6e02a-ca1d-4074-bb22-2b1d7110cb31 service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:20:53.337285 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1273/2554] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:20:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:21:02.548772 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42c9f63-ccbf-4439-9243-3759a6431afd None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:02.550326 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c42c9f63-ccbf-4439-9243-3759a6431afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:02.570650 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42c9f63-ccbf-4439-9243-3759a6431afd None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:02.581056 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89754876-5575-4d9a-8990-1b14ab248479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:02.581976 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89754876-5575-4d9a-8990-1b14ab248479 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:02.582412 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89754876-5575-4d9a-8990-1b14ab248479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:02.582721 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89754876-5575-4d9a-8990-1b14ab248479 admin admin] PATH_INFO: `/v3/users/a64d6fad442b459791ae1988483d1970` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:02.584718 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89754876-5575-4d9a-8990-1b14ab248479 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a64d6fad442b459791ae1988483d1970)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:02.587180 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89754876-5575-4d9a-8990-1b14ab248479 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:02.657667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-89754876-5575-4d9a-8990-1b14ab248479 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a64d6fad442b459791ae1988483d1970'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:02.666386 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-89754876-5575-4d9a-8990-1b14ab248479 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a64d6fad442b459791ae1988483d1970'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:02.667239 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-89754876-5575-4d9a-8990-1b14ab248479 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a64d6fad442b459791ae1988483d1970'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:02.668037 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-89754876-5575-4d9a-8990-1b14ab248479 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a64d6fad442b459791ae1988483d1970'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:02.682275 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-89754876-5575-4d9a-8990-1b14ab248479 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a64d6fad442b459791ae1988483d1970'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:02.705060 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1282/2555] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:21:02 2022] DELETE /identity/v3/users/a64d6fad442b459791ae1988483d1970 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:02.709615 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a585567-0632-4da7-af3c-a54ae1ec6249 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:02.710104 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a585567-0632-4da7-af3c-a54ae1ec6249 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:02.710219 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a585567-0632-4da7-af3c-a54ae1ec6249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:02.730465 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-4a585567-0632-4da7-af3c-a54ae1ec6249 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:02.732630 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-4a585567-0632-4da7-af3c-a54ae1ec6249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:02.789381 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:02.805653 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1274/2556] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:02.815946 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89754876-5575-4d9a-8990-1b14ab248479 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:02.817438 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-89754876-5575-4d9a-8990-1b14ab248479 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:02.876220 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89754876-5575-4d9a-8990-1b14ab248479 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:02.880035 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08043f17-349d-4657-88e7-7e78c984a925 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:02.880647 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08043f17-349d-4657-88e7-7e78c984a925 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:02.880919 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08043f17-349d-4657-88e7-7e78c984a925 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:02.881194 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08043f17-349d-4657-88e7-7e78c984a925 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:02.890954 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08043f17-349d-4657-88e7-7e78c984a925 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:02.892366 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08043f17-349d-4657-88e7-7e78c984a925 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:02.901763 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1283/2557] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:21:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:21:03.254981 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a585567-0632-4da7-af3c-a54ae1ec6249 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:03.262112 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-4a585567-0632-4da7-af3c-a54ae1ec6249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:03.282405 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a585567-0632-4da7-af3c-a54ae1ec6249 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:03.287505 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e885dc4-1c8c-408b-8d23-58f7a493fcfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:03.288467 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e885dc4-1c8c-408b-8d23-58f7a493fcfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:03.289188 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e885dc4-1c8c-408b-8d23-58f7a493fcfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:03.289545 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e885dc4-1c8c-408b-8d23-58f7a493fcfa admin admin] PATH_INFO: `/v3/projects/7dceba7df7bf4c1fad8656561cb63a1f` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:03.290789 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e885dc4-1c8c-408b-8d23-58f7a493fcfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dceba7df7bf4c1fad8656561cb63a1f)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:03.292824 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e885dc4-1c8c-408b-8d23-58f7a493fcfa admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:03.349560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-5e885dc4-1c8c-408b-8d23-58f7a493fcfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dceba7df7bf4c1fad8656561cb63a1f', 'request_id': 'req-5e885dc4-1c8c-408b-8d23-58f7a493fcfa'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:03.355503 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-5e885dc4-1c8c-408b-8d23-58f7a493fcfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dceba7df7bf4c1fad8656561cb63a1f', 'request_id': 'req-5e885dc4-1c8c-408b-8d23-58f7a493fcfa'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:03.363914 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-5e885dc4-1c8c-408b-8d23-58f7a493fcfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dceba7df7bf4c1fad8656561cb63a1f', 'request_id': 'req-5e885dc4-1c8c-408b-8d23-58f7a493fcfa'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:03.381504 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-5e885dc4-1c8c-408b-8d23-58f7a493fcfa admin admin] The token cache is being invalidate because project 7dceba7df7bf4c1fad8656561cb63a1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:21:03.381968 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-5e885dc4-1c8c-408b-8d23-58f7a493fcfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:03.382894 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-5e885dc4-1c8c-408b-8d23-58f7a493fcfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:03.387731 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1275/2558] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:21:03 2022] DELETE /identity/v3/projects/7dceba7df7bf4c1fad8656561cb63a1f => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:03.389630 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08043f17-349d-4657-88e7-7e78c984a925 service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:03.399995 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08043f17-349d-4657-88e7-7e78c984a925 service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:03.434736 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:03.465271 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2bc753-d961-4c78-967a-2d12ba38f0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:03.466020 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2bc753-d961-4c78-967a-2d12ba38f0ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:03.466436 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2bc753-d961-4c78-967a-2d12ba38f0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:03.466811 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2bc753-d961-4c78-967a-2d12ba38f0ce admin admin] PATH_INFO: `/v3/users/89383f692c824ef89d3c118fd82798aa` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:03.468575 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2bc753-d961-4c78-967a-2d12ba38f0ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=89383f692c824ef89d3c118fd82798aa)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:03.470253 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2bc753-d961-4c78-967a-2d12ba38f0ce admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:03.533399 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-ca2bc753-d961-4c78-967a-2d12ba38f0ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89383f692c824ef89d3c118fd82798aa'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:03.542865 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-ca2bc753-d961-4c78-967a-2d12ba38f0ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89383f692c824ef89d3c118fd82798aa'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:03.543747 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-ca2bc753-d961-4c78-967a-2d12ba38f0ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89383f692c824ef89d3c118fd82798aa'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:03.544261 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-ca2bc753-d961-4c78-967a-2d12ba38f0ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89383f692c824ef89d3c118fd82798aa'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:03.551878 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-ca2bc753-d961-4c78-967a-2d12ba38f0ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89383f692c824ef89d3c118fd82798aa'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:03.571562 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1284/2559] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:21:03 2022] DELETE /identity/v3/users/89383f692c824ef89d3c118fd82798aa => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:03.860542 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e885dc4-1c8c-408b-8d23-58f7a493fcfa admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:03.862356 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-5e885dc4-1c8c-408b-8d23-58f7a493fcfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:03.879927 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e885dc4-1c8c-408b-8d23-58f7a493fcfa admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:03.941150 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36cf0c-98b4-425f-8b1f-e509f39dfe44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:03.941786 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36cf0c-98b4-425f-8b1f-e509f39dfe44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:03.942596 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36cf0c-98b4-425f-8b1f-e509f39dfe44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:03.942596 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36cf0c-98b4-425f-8b1f-e509f39dfe44 admin admin] PATH_INFO: `/v3/projects/3dc3326fe370446d8f941d4f2aaa411f` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:03.943505 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f36cf0c-98b4-425f-8b1f-e509f39dfe44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dc3326fe370446d8f941d4f2aaa411f)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:03.945628 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f36cf0c-98b4-425f-8b1f-e509f39dfe44 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:03.996741 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-5f36cf0c-98b4-425f-8b1f-e509f39dfe44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dc3326fe370446d8f941d4f2aaa411f', 'request_id': 'req-5f36cf0c-98b4-425f-8b1f-e509f39dfe44'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:04.004171 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-5f36cf0c-98b4-425f-8b1f-e509f39dfe44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dc3326fe370446d8f941d4f2aaa411f', 'request_id': 'req-5f36cf0c-98b4-425f-8b1f-e509f39dfe44'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:04.014399 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-5f36cf0c-98b4-425f-8b1f-e509f39dfe44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dc3326fe370446d8f941d4f2aaa411f', 'request_id': 'req-5f36cf0c-98b4-425f-8b1f-e509f39dfe44'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:04.026531 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-5f36cf0c-98b4-425f-8b1f-e509f39dfe44 admin admin] The token cache is being invalidate because project 3dc3326fe370446d8f941d4f2aaa411f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:21:04.026824 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-5f36cf0c-98b4-425f-8b1f-e509f39dfe44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:04.027537 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-5f36cf0c-98b4-425f-8b1f-e509f39dfe44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:04.029786 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1276/2560] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:21:03 2022] DELETE /identity/v3/projects/3dc3326fe370446d8f941d4f2aaa411f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:04.035383 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c999495-bf4e-4d95-b94c-e56ec29d45c1 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:04.035676 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c999495-bf4e-4d95-b94c-e56ec29d45c1 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:04.035970 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c999495-bf4e-4d95-b94c-e56ec29d45c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:04.055988 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-9c999495-bf4e-4d95-b94c-e56ec29d45c1 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:04.057181 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-9c999495-bf4e-4d95-b94c-e56ec29d45c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:04.114922 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1285/2561] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:04.123109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b146bc7-7f30-4887-bbff-233e0e3e191c None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:04.123444 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b146bc7-7f30-4887-bbff-233e0e3e191c None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:04.123720 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b146bc7-7f30-4887-bbff-233e0e3e191c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:04.132898 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:04.145213 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-9b146bc7-7f30-4887-bbff-233e0e3e191c None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:04.146402 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-9b146bc7-7f30-4887-bbff-233e0e3e191c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:04.154947 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1277/2562] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:04.159024 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c999495-bf4e-4d95-b94c-e56ec29d45c1 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:04.168387 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c999495-bf4e-4d95-b94c-e56ec29d45c1 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:04.173526 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a6b596-25ac-4776-b86b-4537d0c6b264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:04.174115 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6b596-25ac-4776-b86b-4537d0c6b264 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:04.174391 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6b596-25ac-4776-b86b-4537d0c6b264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:04.174660 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6b596-25ac-4776-b86b-4537d0c6b264 admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:04.175326 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a6b596-25ac-4776-b86b-4537d0c6b264 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:21:04.175623 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a6b596-25ac-4776-b86b-4537d0c6b264 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:04.177189 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a6b596-25ac-4776-b86b-4537d0c6b264 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:04.188814 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1286/2563] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:21:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:21:04.192849 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b146bc7-7f30-4887-bbff-233e0e3e191c None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:04.201799 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b146bc7-7f30-4887-bbff-233e0e3e191c None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:04.206366 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd736fdc-d6a2-473a-ba15-a0300daec861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:04.206991 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd736fdc-d6a2-473a-ba15-a0300daec861 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:04.207264 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd736fdc-d6a2-473a-ba15-a0300daec861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:04.207529 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd736fdc-d6a2-473a-ba15-a0300daec861 admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:04.208153 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd736fdc-d6a2-473a-ba15-a0300daec861 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:04.210012 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd736fdc-d6a2-473a-ba15-a0300daec861 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:04.237599 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1278/2564] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:21:04 2022] POST /identity/v3/projects => generated 389 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:21:04.241515 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a6b596-25ac-4776-b86b-4537d0c6b264 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:04.251249 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a6b596-25ac-4776-b86b-4537d0c6b264 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:04.291300 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:04.307279 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebad82e-7991-40ec-ad7f-36c51b704842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:04.307871 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebad82e-7991-40ec-ad7f-36c51b704842 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:04.308145 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebad82e-7991-40ec-ad7f-36c51b704842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:04.308411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebad82e-7991-40ec-ad7f-36c51b704842 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:04.309023 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebad82e-7991-40ec-ad7f-36c51b704842 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:04.311231 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebad82e-7991-40ec-ad7f-36c51b704842 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:04.330782 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1287/2565] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:21:04 2022] POST /identity/v3/users => generated 295 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:21:04.334981 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd736fdc-d6a2-473a-ba15-a0300daec861 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:04.345906 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd736fdc-d6a2-473a-ba15-a0300daec861 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:04.350793 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b0d6e2-1234-4f34-8add-86d34ee5c4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:04.351420 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b0d6e2-1234-4f34-8add-86d34ee5c4b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:04.351694 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b0d6e2-1234-4f34-8add-86d34ee5c4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:04.351962 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b0d6e2-1234-4f34-8add-86d34ee5c4b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:04.352594 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b0d6e2-1234-4f34-8add-86d34ee5c4b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:04.354171 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b0d6e2-1234-4f34-8add-86d34ee5c4b5 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:04.364817 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1279/2566] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:21:04.368625 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebad82e-7991-40ec-ad7f-36c51b704842 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:04.378214 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebad82e-7991-40ec-ad7f-36c51b704842 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:04.382655 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68e6b00-b6b0-4743-8e37-af7be06adf8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:04.383264 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e6b00-b6b0-4743-8e37-af7be06adf8f admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:04.383540 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e6b00-b6b0-4743-8e37-af7be06adf8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:04.383806 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e6b00-b6b0-4743-8e37-af7be06adf8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:04.384431 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68e6b00-b6b0-4743-8e37-af7be06adf8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:04.385972 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68e6b00-b6b0-4743-8e37-af7be06adf8f admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:04.396579 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1288/2567] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:21:04.400452 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b0d6e2-1234-4f34-8add-86d34ee5c4b5 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:04.411031 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b0d6e2-1234-4f34-8add-86d34ee5c4b5 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:04.415633 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bafdbd0-c92f-4ded-bbc6-0e5cc8ff5bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:04.416269 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafdbd0-c92f-4ded-bbc6-0e5cc8ff5bf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:04.416543 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafdbd0-c92f-4ded-bbc6-0e5cc8ff5bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:04.416815 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafdbd0-c92f-4ded-bbc6-0e5cc8ff5bf0 admin admin] PATH_INFO: `/v3/projects/b7448f7fb54a4089bdcb982c0d335329/users/507b659b6bd042ec80ab4d3bae829aa7/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:04.430763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bafdbd0-c92f-4ded-bbc6-0e5cc8ff5bf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7448f7fb54a4089bdcb982c0d335329, user_id=507b659b6bd042ec80ab4d3bae829aa7, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:04.433388 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bafdbd0-c92f-4ded-bbc6-0e5cc8ff5bf0 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:04.446352 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1280/2568] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:21:04 2022] PUT /identity/v3/projects/b7448f7fb54a4089bdcb982c0d335329/users/507b659b6bd042ec80ab4d3bae829aa7/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:04.456903 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e770b-7601-4c6f-b0d0-9da2b85636ea None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:04.458207 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e770b-7601-4c6f-b0d0-9da2b85636ea None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:04.458207 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e770b-7601-4c6f-b0d0-9da2b85636ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:04.485371 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-871e770b-7601-4c6f-b0d0-9da2b85636ea None None] MFA Rules not processed for user `507b659b6bd042ec80ab4d3bae829aa7`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:04.486585 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-871e770b-7601-4c6f-b0d0-9da2b85636ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:04.561819 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1289/2569] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:04 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:21:04.566149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bafdbd0-c92f-4ded-bbc6-0e5cc8ff5bf0 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:04.678530 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bafdbd0-c92f-4ded-bbc6-0e5cc8ff5bf0 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:04.686202 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99586c42-dd5a-428f-bfdb-3226031d19dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:04.686756 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99586c42-dd5a-428f-bfdb-3226031d19dd admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:04.687178 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99586c42-dd5a-428f-bfdb-3226031d19dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:04.687512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99586c42-dd5a-428f-bfdb-3226031d19dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:04.688243 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99586c42-dd5a-428f-bfdb-3226031d19dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:04.690308 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99586c42-dd5a-428f-bfdb-3226031d19dd admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:04.745832 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1281/2570] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:21:04 2022] POST /identity/v3/projects => generated 389 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:21:04.749594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e770b-7601-4c6f-b0d0-9da2b85636ea None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:04.762551 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e770b-7601-4c6f-b0d0-9da2b85636ea None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:04.858872 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbad9cc-7aa2-411e-85b6-020c9a7687f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:04.858872 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbad9cc-7aa2-411e-85b6-020c9a7687f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:04.858872 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbad9cc-7aa2-411e-85b6-020c9a7687f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:04.858872 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbad9cc-7aa2-411e-85b6-020c9a7687f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:04.860130 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbad9cc-7aa2-411e-85b6-020c9a7687f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:04.860130 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbad9cc-7aa2-411e-85b6-020c9a7687f7 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:04.881268 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1290/2571] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:21:04 2022] POST /identity/v3/users => generated 301 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:21:04.885752 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99586c42-dd5a-428f-bfdb-3226031d19dd admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:04.898711 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99586c42-dd5a-428f-bfdb-3226031d19dd admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:04.904954 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc12c2-9405-489f-acfe-46773a571355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:04.905744 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc12c2-9405-489f-acfe-46773a571355 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:04.906037 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc12c2-9405-489f-acfe-46773a571355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:04.906374 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc12c2-9405-489f-acfe-46773a571355 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:04.907278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dc12c2-9405-489f-acfe-46773a571355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:04.908877 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dc12c2-9405-489f-acfe-46773a571355 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:04.920332 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1282/2572] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:21:04.924605 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbad9cc-7aa2-411e-85b6-020c9a7687f7 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:04.934348 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbad9cc-7aa2-411e-85b6-020c9a7687f7 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:04.939726 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20622d0-25d5-48d7-9f52-6977de06c694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:04.940368 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20622d0-25d5-48d7-9f52-6977de06c694 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:04.940644 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20622d0-25d5-48d7-9f52-6977de06c694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:04.940912 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20622d0-25d5-48d7-9f52-6977de06c694 admin admin] PATH_INFO: `/v3/projects/110f2c14260044a2a8177fb583ec0075/users/8224e9c5d7594ee4a43d01e7ae2fb5df/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:04.957006 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20622d0-25d5-48d7-9f52-6977de06c694 admin admin] RBAC: Authorizing `identity:create_grant(project_id=110f2c14260044a2a8177fb583ec0075, user_id=8224e9c5d7594ee4a43d01e7ae2fb5df, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:04.959651 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20622d0-25d5-48d7-9f52-6977de06c694 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:04.973016 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1291/2573] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:21:04 2022] PUT /identity/v3/projects/110f2c14260044a2a8177fb583ec0075/users/8224e9c5d7594ee4a43d01e7ae2fb5df/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:04.977438 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc12c2-9405-489f-acfe-46773a571355 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:04.987755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc12c2-9405-489f-acfe-46773a571355 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:05.045324 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f00a858-c7c6-4be9-b1b8-dc11bf7c7766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:05.045993 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f00a858-c7c6-4be9-b1b8-dc11bf7c7766 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:05.046276 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f00a858-c7c6-4be9-b1b8-dc11bf7c7766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:05.046547 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f00a858-c7c6-4be9-b1b8-dc11bf7c7766 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:05.047227 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f00a858-c7c6-4be9-b1b8-dc11bf7c7766 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:05.048807 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f00a858-c7c6-4be9-b1b8-dc11bf7c7766 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:05.058975 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1283/2574] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:21:05.063889 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20622d0-25d5-48d7-9f52-6977de06c694 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:05.074181 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20622d0-25d5-48d7-9f52-6977de06c694 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:05.079562 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7862257-2c73-4f57-a5b5-2d4249045076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:05.080218 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7862257-2c73-4f57-a5b5-2d4249045076 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:05.080560 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7862257-2c73-4f57-a5b5-2d4249045076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:05.080899 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7862257-2c73-4f57-a5b5-2d4249045076 admin admin] PATH_INFO: `/v3/projects/110f2c14260044a2a8177fb583ec0075/users/8224e9c5d7594ee4a43d01e7ae2fb5df/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:05.083004 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7862257-2c73-4f57-a5b5-2d4249045076 admin admin] RBAC: Authorizing `identity:create_grant(project_id=110f2c14260044a2a8177fb583ec0075, user_id=8224e9c5d7594ee4a43d01e7ae2fb5df, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:05.085687 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7862257-2c73-4f57-a5b5-2d4249045076 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:05.096631 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1292/2575] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:21:05 2022] PUT /identity/v3/projects/110f2c14260044a2a8177fb583ec0075/users/8224e9c5d7594ee4a43d01e7ae2fb5df/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:21:05.107879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555c5a7-746c-46ae-92f4-7c1abfe58ad6 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:05.108167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555c5a7-746c-46ae-92f4-7c1abfe58ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:05.108440 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555c5a7-746c-46ae-92f4-7c1abfe58ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:05.129880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:05.138912 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-6555c5a7-746c-46ae-92f4-7c1abfe58ad6 None None] MFA Rules not processed for user `8224e9c5d7594ee4a43d01e7ae2fb5df`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:05.140289 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-6555c5a7-746c-46ae-92f4-7c1abfe58ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:05.214557 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1284/2576] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:05 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:05.225049 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7862257-2c73-4f57-a5b5-2d4249045076 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:05.226736 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b7862257-2c73-4f57-a5b5-2d4249045076 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:05.234042 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:05.286086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7862257-2c73-4f57-a5b5-2d4249045076 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:05.290431 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f00a8b3-4113-4e2b-88ad-6e5de7511336 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:05.291065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00a8b3-4113-4e2b-88ad-6e5de7511336 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:05.291415 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00a8b3-4113-4e2b-88ad-6e5de7511336 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:05.291765 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00a8b3-4113-4e2b-88ad-6e5de7511336 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:05.302023 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f00a8b3-4113-4e2b-88ad-6e5de7511336 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:05.303469 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f00a8b3-4113-4e2b-88ad-6e5de7511336 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:05.313618 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1293/2577] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:21:05 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:21:05.452739 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555c5a7-746c-46ae-92f4-7c1abfe58ad6 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:05.531237 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555c5a7-746c-46ae-92f4-7c1abfe58ad6 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:05.535731 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4b02f-eced-4242-8ee9-72ec05452494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:05.536357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4b02f-eced-4242-8ee9-72ec05452494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:05.536633 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4b02f-eced-4242-8ee9-72ec05452494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:05.536906 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4b02f-eced-4242-8ee9-72ec05452494 admin admin] PATH_INFO: `/v3/users/507b659b6bd042ec80ab4d3bae829aa7` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:05.538055 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4b02f-eced-4242-8ee9-72ec05452494 admin admin] RBAC: Authorizing `identity:delete_user(user_id=507b659b6bd042ec80ab4d3bae829aa7)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:05.539994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4b02f-eced-4242-8ee9-72ec05452494 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:05.585439 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-1aa4b02f-eced-4242-8ee9-72ec05452494 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '507b659b6bd042ec80ab4d3bae829aa7'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:05.593326 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-1aa4b02f-eced-4242-8ee9-72ec05452494 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '507b659b6bd042ec80ab4d3bae829aa7'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:05.594035 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-1aa4b02f-eced-4242-8ee9-72ec05452494 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '507b659b6bd042ec80ab4d3bae829aa7'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:05.594635 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-1aa4b02f-eced-4242-8ee9-72ec05452494 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '507b659b6bd042ec80ab4d3bae829aa7'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:05.601880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-1aa4b02f-eced-4242-8ee9-72ec05452494 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '507b659b6bd042ec80ab4d3bae829aa7'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:05.619967 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1285/2578] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:21:05 2022] DELETE /identity/v3/users/507b659b6bd042ec80ab4d3bae829aa7 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:05.625369 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09a334-83d9-4f48-b1a5-5c64f544c807 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:05.626040 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09a334-83d9-4f48-b1a5-5c64f544c807 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:05.626251 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09a334-83d9-4f48-b1a5-5c64f544c807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:05.645583 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-8c09a334-83d9-4f48-b1a5-5c64f544c807 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:05.646766 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-8c09a334-83d9-4f48-b1a5-5c64f544c807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:05.705690 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1294/2579] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:05.715429 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4b02f-eced-4242-8ee9-72ec05452494 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:05.717364 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-1aa4b02f-eced-4242-8ee9-72ec05452494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:05.735646 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:05.775691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4b02f-eced-4242-8ee9-72ec05452494 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:05.783625 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad04a2ef-3c79-4e63-baef-3e94608889f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:05.784243 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04a2ef-3c79-4e63-baef-3e94608889f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:05.784522 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04a2ef-3c79-4e63-baef-3e94608889f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:05.784796 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04a2ef-3c79-4e63-baef-3e94608889f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:05.796554 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad04a2ef-3c79-4e63-baef-3e94608889f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:05.797980 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad04a2ef-3c79-4e63-baef-3e94608889f9 service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:05.807574 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1286/2580] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:21:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:21:06.088527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09a334-83d9-4f48-b1a5-5c64f544c807 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:06.089909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-8c09a334-83d9-4f48-b1a5-5c64f544c807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:06.102512 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09a334-83d9-4f48-b1a5-5c64f544c807 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:06.107207 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b021ce-3c60-4dfa-b73e-9eb7d7537e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:06.107844 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b021ce-3c60-4dfa-b73e-9eb7d7537e83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:06.108126 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b021ce-3c60-4dfa-b73e-9eb7d7537e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:06.108398 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b021ce-3c60-4dfa-b73e-9eb7d7537e83 admin admin] PATH_INFO: `/v3/projects/b7448f7fb54a4089bdcb982c0d335329` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:06.109526 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b021ce-3c60-4dfa-b73e-9eb7d7537e83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7448f7fb54a4089bdcb982c0d335329)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:06.111444 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b021ce-3c60-4dfa-b73e-9eb7d7537e83 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:06.158395 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-19b021ce-3c60-4dfa-b73e-9eb7d7537e83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7448f7fb54a4089bdcb982c0d335329', 'request_id': 'req-19b021ce-3c60-4dfa-b73e-9eb7d7537e83'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:06.165440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-19b021ce-3c60-4dfa-b73e-9eb7d7537e83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7448f7fb54a4089bdcb982c0d335329', 'request_id': 'req-19b021ce-3c60-4dfa-b73e-9eb7d7537e83'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:06.171028 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-19b021ce-3c60-4dfa-b73e-9eb7d7537e83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7448f7fb54a4089bdcb982c0d335329', 'request_id': 'req-19b021ce-3c60-4dfa-b73e-9eb7d7537e83'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:06.182017 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-19b021ce-3c60-4dfa-b73e-9eb7d7537e83 admin admin] The token cache is being invalidate because project b7448f7fb54a4089bdcb982c0d335329 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:21:06.182303 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-19b021ce-3c60-4dfa-b73e-9eb7d7537e83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:06.182963 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-19b021ce-3c60-4dfa-b73e-9eb7d7537e83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:06.184421 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1295/2581] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:21:06 2022] DELETE /identity/v3/projects/b7448f7fb54a4089bdcb982c0d335329 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:06.188643 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad04a2ef-3c79-4e63-baef-3e94608889f9 service neutron] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:06.198235 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad04a2ef-3c79-4e63-baef-3e94608889f9 service neutron] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:06.253921 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bbcd0c-40db-45c1-9796-f9f7b44a57d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:06.254556 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbcd0c-40db-45c1-9796-f9f7b44a57d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:06.254855 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbcd0c-40db-45c1-9796-f9f7b44a57d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:06.255137 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbcd0c-40db-45c1-9796-f9f7b44a57d4 admin admin] PATH_INFO: `/v3/users/8224e9c5d7594ee4a43d01e7ae2fb5df` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:06.256248 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bbcd0c-40db-45c1-9796-f9f7b44a57d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8224e9c5d7594ee4a43d01e7ae2fb5df)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:06.258348 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bbcd0c-40db-45c1-9796-f9f7b44a57d4 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:06.302043 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-48bbcd0c-40db-45c1-9796-f9f7b44a57d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8224e9c5d7594ee4a43d01e7ae2fb5df'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:06.310589 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-48bbcd0c-40db-45c1-9796-f9f7b44a57d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8224e9c5d7594ee4a43d01e7ae2fb5df'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:06.311352 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-48bbcd0c-40db-45c1-9796-f9f7b44a57d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8224e9c5d7594ee4a43d01e7ae2fb5df'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:06.311963 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-48bbcd0c-40db-45c1-9796-f9f7b44a57d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8224e9c5d7594ee4a43d01e7ae2fb5df'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:06.318893 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-48bbcd0c-40db-45c1-9796-f9f7b44a57d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8224e9c5d7594ee4a43d01e7ae2fb5df'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:06.337624 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1287/2582] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:21:06 2022] DELETE /identity/v3/users/8224e9c5d7594ee4a43d01e7ae2fb5df => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:06.619264 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b021ce-3c60-4dfa-b73e-9eb7d7537e83 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:06.620752 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-19b021ce-3c60-4dfa-b73e-9eb7d7537e83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:06.646404 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b021ce-3c60-4dfa-b73e-9eb7d7537e83 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:06.680607 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:06.717769 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15f471b-881b-4c5b-ab36-544dc7e64381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:06.718409 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f471b-881b-4c5b-ab36-544dc7e64381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:06.718685 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f471b-881b-4c5b-ab36-544dc7e64381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:06.718997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f471b-881b-4c5b-ab36-544dc7e64381 admin admin] PATH_INFO: `/v3/projects/110f2c14260044a2a8177fb583ec0075` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:06.720139 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15f471b-881b-4c5b-ab36-544dc7e64381 admin admin] RBAC: Authorizing `identity:delete_project(project_id=110f2c14260044a2a8177fb583ec0075)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:06.722056 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15f471b-881b-4c5b-ab36-544dc7e64381 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:06.774418 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c15f471b-881b-4c5b-ab36-544dc7e64381 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '110f2c14260044a2a8177fb583ec0075', 'request_id': 'req-c15f471b-881b-4c5b-ab36-544dc7e64381'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:06.781409 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c15f471b-881b-4c5b-ab36-544dc7e64381 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '110f2c14260044a2a8177fb583ec0075', 'request_id': 'req-c15f471b-881b-4c5b-ab36-544dc7e64381'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:06.787676 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c15f471b-881b-4c5b-ab36-544dc7e64381 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '110f2c14260044a2a8177fb583ec0075', 'request_id': 'req-c15f471b-881b-4c5b-ab36-544dc7e64381'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:06.800513 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c15f471b-881b-4c5b-ab36-544dc7e64381 admin admin] The token cache is being invalidate because project 110f2c14260044a2a8177fb583ec0075 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:21:06.800804 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c15f471b-881b-4c5b-ab36-544dc7e64381 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:06.801480 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c15f471b-881b-4c5b-ab36-544dc7e64381 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:06.803105 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1296/2583] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:21:06 2022] DELETE /identity/v3/projects/110f2c14260044a2a8177fb583ec0075 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:06.808210 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02d192-2462-4843-826e-907ceab9d196 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:06.808616 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02d192-2462-4843-826e-907ceab9d196 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:06.808966 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02d192-2462-4843-826e-907ceab9d196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:06.829315 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-8f02d192-2462-4843-826e-907ceab9d196 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:06.830599 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-8f02d192-2462-4843-826e-907ceab9d196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:06.864511 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:06.893492 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1288/2584] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:06.902433 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382cb1c-7db5-4ddc-9db4-149bcec44959 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:06.902780 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382cb1c-7db5-4ddc-9db4-149bcec44959 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:06.903129 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382cb1c-7db5-4ddc-9db4-149bcec44959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:06.924383 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-7382cb1c-7db5-4ddc-9db4-149bcec44959 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:06.925597 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7382cb1c-7db5-4ddc-9db4-149bcec44959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:06.934392 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1297/2585] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:06.939458 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02d192-2462-4843-826e-907ceab9d196 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:06.949515 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02d192-2462-4843-826e-907ceab9d196 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:06.954313 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aeb13d-5028-4ef0-a488-f86834c383cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:06.954941 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aeb13d-5028-4ef0-a488-f86834c383cc admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:06.955221 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aeb13d-5028-4ef0-a488-f86834c383cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:06.955513 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aeb13d-5028-4ef0-a488-f86834c383cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:06.956167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aeb13d-5028-4ef0-a488-f86834c383cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:21:06.956456 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aeb13d-5028-4ef0-a488-f86834c383cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:06.958044 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aeb13d-5028-4ef0-a488-f86834c383cc admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:06.969521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1289/2586] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:21:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:21:06.973724 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382cb1c-7db5-4ddc-9db4-149bcec44959 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:06.983374 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382cb1c-7db5-4ddc-9db4-149bcec44959 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:06.987943 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6cfb91-dfaf-4ea6-9da4-cb69b7643a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:06.988528 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6cfb91-dfaf-4ea6-9da4-cb69b7643a3d admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:06.988803 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6cfb91-dfaf-4ea6-9da4-cb69b7643a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:06.989073 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6cfb91-dfaf-4ea6-9da4-cb69b7643a3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:06.989690 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6cfb91-dfaf-4ea6-9da4-cb69b7643a3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:06.991593 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6cfb91-dfaf-4ea6-9da4-cb69b7643a3d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:07.018435 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1298/2587] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:21:06 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:21:07.022507 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aeb13d-5028-4ef0-a488-f86834c383cc admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:07.032905 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aeb13d-5028-4ef0-a488-f86834c383cc admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:07.090571 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd540e7b-93e0-4466-876a-15adbc5935e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:07.091701 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd540e7b-93e0-4466-876a-15adbc5935e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:07.091978 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd540e7b-93e0-4466-876a-15adbc5935e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:07.092248 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd540e7b-93e0-4466-876a-15adbc5935e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:07.092875 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd540e7b-93e0-4466-876a-15adbc5935e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:07.094778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd540e7b-93e0-4466-876a-15adbc5935e6 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:07.122907 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1290/2588] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:21:07 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:21:07.124021 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6cfb91-dfaf-4ea6-9da4-cb69b7643a3d admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:07.134378 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6cfb91-dfaf-4ea6-9da4-cb69b7643a3d admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:07.139383 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1434ca29-3455-4392-877b-69d460f1bfb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:07.139969 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434ca29-3455-4392-877b-69d460f1bfb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:07.140241 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434ca29-3455-4392-877b-69d460f1bfb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:07.140513 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434ca29-3455-4392-877b-69d460f1bfb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:07.141149 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1434ca29-3455-4392-877b-69d460f1bfb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:07.142771 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1434ca29-3455-4392-877b-69d460f1bfb1 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:07.153189 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1299/2589] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:21:07.157706 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd540e7b-93e0-4466-876a-15adbc5935e6 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:07.167813 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd540e7b-93e0-4466-876a-15adbc5935e6 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:07.173199 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306f4dc-6a9e-4f04-a801-721a1eb68d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:07.173827 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306f4dc-6a9e-4f04-a801-721a1eb68d90 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:07.174105 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306f4dc-6a9e-4f04-a801-721a1eb68d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:07.174371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306f4dc-6a9e-4f04-a801-721a1eb68d90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:07.175039 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306f4dc-6a9e-4f04-a801-721a1eb68d90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:07.176786 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306f4dc-6a9e-4f04-a801-721a1eb68d90 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:07.189552 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1291/2590] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:21:07.195034 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1434ca29-3455-4392-877b-69d460f1bfb1 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:07.205275 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1434ca29-3455-4392-877b-69d460f1bfb1 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:07.209814 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3bc2a-62a3-4081-b797-a10c7619c701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:07.210459 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3bc2a-62a3-4081-b797-a10c7619c701 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:07.210737 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3bc2a-62a3-4081-b797-a10c7619c701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:07.211046 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3bc2a-62a3-4081-b797-a10c7619c701 admin admin] PATH_INFO: `/v3/projects/0412ec3ff65b4c7284b4aeba06d355dc/users/3877b0d0d97743fb91fc3c2e8ba4006a/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:07.225800 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e3bc2a-62a3-4081-b797-a10c7619c701 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0412ec3ff65b4c7284b4aeba06d355dc, user_id=3877b0d0d97743fb91fc3c2e8ba4006a, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:07.228465 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e3bc2a-62a3-4081-b797-a10c7619c701 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:07.241747 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1300/2591] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:21:07 2022] PUT /identity/v3/projects/0412ec3ff65b4c7284b4aeba06d355dc/users/3877b0d0d97743fb91fc3c2e8ba4006a/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:07.246861 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de473e3e-08a8-4c93-80f3-ca48b69d2143 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:07.247162 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de473e3e-08a8-4c93-80f3-ca48b69d2143 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:07.247608 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de473e3e-08a8-4c93-80f3-ca48b69d2143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:07.268987 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-de473e3e-08a8-4c93-80f3-ca48b69d2143 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:07.270199 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-de473e3e-08a8-4c93-80f3-ca48b69d2143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:07.332818 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1292/2592] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:21:07.344648 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3bc2a-62a3-4081-b797-a10c7619c701 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:07.346373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b0e3bc2a-62a3-4081-b797-a10c7619c701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:07.408729 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3bc2a-62a3-4081-b797-a10c7619c701 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:07.412404 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab35bf-2971-45d0-9c3d-193c7bbf052f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:07.413012 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab35bf-2971-45d0-9c3d-193c7bbf052f service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:07.413407 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab35bf-2971-45d0-9c3d-193c7bbf052f service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:07.413549 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab35bf-2971-45d0-9c3d-193c7bbf052f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:07.422819 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab35bf-2971-45d0-9c3d-193c7bbf052f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:07.424250 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab35bf-2971-45d0-9c3d-193c7bbf052f service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:07.429558 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:07.434214 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1301/2593] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:21:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:21:07.924301 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de473e3e-08a8-4c93-80f3-ca48b69d2143 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:07.925612 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-de473e3e-08a8-4c93-80f3-ca48b69d2143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:07.938441 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de473e3e-08a8-4c93-80f3-ca48b69d2143 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:07.942187 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03318714-ab13-4fff-9dd8-84bc7c035c95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:07.942792 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03318714-ab13-4fff-9dd8-84bc7c035c95 service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:07.943095 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03318714-ab13-4fff-9dd8-84bc7c035c95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:07.943829 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03318714-ab13-4fff-9dd8-84bc7c035c95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:07.945134 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-03318714-ab13-4fff-9dd8-84bc7c035c95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:07.957759 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03318714-ab13-4fff-9dd8-84bc7c035c95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:07.959217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03318714-ab13-4fff-9dd8-84bc7c035c95 service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:07.969468 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1293/2594] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:21:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:21:08.409440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319615c9-83bc-4f64-81c9-70a0dec74cc8 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:08.409975 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319615c9-83bc-4f64-81c9-70a0dec74cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:08.410260 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319615c9-83bc-4f64-81c9-70a0dec74cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:08.449675 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-319615c9-83bc-4f64-81c9-70a0dec74cc8 None None] MFA Rules not processed for user `3877b0d0d97743fb91fc3c2e8ba4006a`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:08.451382 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-319615c9-83bc-4f64-81c9-70a0dec74cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:08.529264 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1302/2595] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:21:08.533847 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03318714-ab13-4fff-9dd8-84bc7c035c95 service neutron] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:08.546337 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03318714-ab13-4fff-9dd8-84bc7c035c95 service neutron] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:08.551290 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6963badb-8bff-4994-8ae4-8037bf632c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:08.551970 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963badb-8bff-4994-8ae4-8037bf632c47 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:08.552322 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963badb-8bff-4994-8ae4-8037bf632c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:08.552668 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963badb-8bff-4994-8ae4-8037bf632c47 admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:08.553384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6963badb-8bff-4994-8ae4-8037bf632c47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:08.556044 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6963badb-8bff-4994-8ae4-8037bf632c47 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:08.586909 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1294/2596] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:21:08 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:21:08.590857 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319615c9-83bc-4f64-81c9-70a0dec74cc8 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:08.601941 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319615c9-83bc-4f64-81c9-70a0dec74cc8 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:08.661829 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff68904-76a5-4359-8897-fd0a764abef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:08.662362 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff68904-76a5-4359-8897-fd0a764abef2 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:08.662676 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff68904-76a5-4359-8897-fd0a764abef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:08.662985 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff68904-76a5-4359-8897-fd0a764abef2 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:08.663620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff68904-76a5-4359-8897-fd0a764abef2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:08.665532 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff68904-76a5-4359-8897-fd0a764abef2 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:08.688102 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1303/2597] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:21:08 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:21:08.692043 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6963badb-8bff-4994-8ae4-8037bf632c47 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:08.702628 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6963badb-8bff-4994-8ae4-8037bf632c47 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:08.707292 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150a1d2-8212-401d-9d7c-5c1205de2bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:08.707917 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150a1d2-8212-401d-9d7c-5c1205de2bad admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:08.708196 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150a1d2-8212-401d-9d7c-5c1205de2bad admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:08.708467 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150a1d2-8212-401d-9d7c-5c1205de2bad admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:08.709113 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7150a1d2-8212-401d-9d7c-5c1205de2bad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:08.710694 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7150a1d2-8212-401d-9d7c-5c1205de2bad admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:08.713274 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:08.723438 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1295/2598] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:21:08.727581 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff68904-76a5-4359-8897-fd0a764abef2 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:08.738324 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff68904-76a5-4359-8897-fd0a764abef2 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:08.742998 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1b7ef-cdfd-47cd-b4a4-6b3c2572a3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:08.743648 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1b7ef-cdfd-47cd-b4a4-6b3c2572a3b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:08.743927 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1b7ef-cdfd-47cd-b4a4-6b3c2572a3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:08.744200 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1b7ef-cdfd-47cd-b4a4-6b3c2572a3b1 admin admin] PATH_INFO: `/v3/projects/16fd2599e7514acf9864bf0bca9ac5f0/users/140a107f2a4148c2acf5a56dc7d64355/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:08.759463 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b1b7ef-cdfd-47cd-b4a4-6b3c2572a3b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16fd2599e7514acf9864bf0bca9ac5f0, user_id=140a107f2a4148c2acf5a56dc7d64355, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:08.762091 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b1b7ef-cdfd-47cd-b4a4-6b3c2572a3b1 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:08.775969 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1304/2599] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:21:08 2022] PUT /identity/v3/projects/16fd2599e7514acf9864bf0bca9ac5f0/users/140a107f2a4148c2acf5a56dc7d64355/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:08.780537 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150a1d2-8212-401d-9d7c-5c1205de2bad admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:08.790789 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150a1d2-8212-401d-9d7c-5c1205de2bad admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:08.848369 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f501a9c-9b45-48d1-95a6-fc44c2ea9d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:08.848989 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f501a9c-9b45-48d1-95a6-fc44c2ea9d3e admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:08.849268 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f501a9c-9b45-48d1-95a6-fc44c2ea9d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:08.849542 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f501a9c-9b45-48d1-95a6-fc44c2ea9d3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:08.850408 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f501a9c-9b45-48d1-95a6-fc44c2ea9d3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:08.852186 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f501a9c-9b45-48d1-95a6-fc44c2ea9d3e admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:08.865676 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1296/2600] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:08 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:21:08.867144 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1b7ef-cdfd-47cd-b4a4-6b3c2572a3b1 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:08.877963 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1b7ef-cdfd-47cd-b4a4-6b3c2572a3b1 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:08.882565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df3fdb0-f692-4467-8985-133c4d70f4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:08.883258 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3fdb0-f692-4467-8985-133c4d70f4f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:08.883552 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3fdb0-f692-4467-8985-133c4d70f4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:08.883810 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3fdb0-f692-4467-8985-133c4d70f4f4 admin admin] PATH_INFO: `/v3/projects/16fd2599e7514acf9864bf0bca9ac5f0/users/140a107f2a4148c2acf5a56dc7d64355/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:08.885693 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df3fdb0-f692-4467-8985-133c4d70f4f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16fd2599e7514acf9864bf0bca9ac5f0, user_id=140a107f2a4148c2acf5a56dc7d64355, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:08.888828 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df3fdb0-f692-4467-8985-133c4d70f4f4 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:08.899068 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1305/2601] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:21:08 2022] PUT /identity/v3/projects/16fd2599e7514acf9864bf0bca9ac5f0/users/140a107f2a4148c2acf5a56dc7d64355/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:21:09.924362 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3df256-f947-49ae-bb92-1c0dc64b2f76 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:09.924862 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3df256-f947-49ae-bb92-1c0dc64b2f76 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:09.924962 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3df256-f947-49ae-bb92-1c0dc64b2f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:09.966611 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-5e3df256-f947-49ae-bb92-1c0dc64b2f76 None None] MFA Rules not processed for user `140a107f2a4148c2acf5a56dc7d64355`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:09.968003 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-5e3df256-f947-49ae-bb92-1c0dc64b2f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:10.050950 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1297/2602] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:09 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:10.062581 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df3fdb0-f692-4467-8985-133c4d70f4f4 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:10.064934 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-9df3fdb0-f692-4467-8985-133c4d70f4f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:10.178055 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:10.194927 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df3fdb0-f692-4467-8985-133c4d70f4f4 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:10.199466 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c71ea-ae04-41bb-ba86-29d41fa6fca6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:10.200396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c71ea-ae04-41bb-ba86-29d41fa6fca6 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:10.200932 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c71ea-ae04-41bb-ba86-29d41fa6fca6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:10.201289 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c71ea-ae04-41bb-ba86-29d41fa6fca6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:10.286992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c71ea-ae04-41bb-ba86-29d41fa6fca6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:10.288566 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c71ea-ae04-41bb-ba86-29d41fa6fca6 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:10.305858 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1306/2603] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:21:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:21:11.718901 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a1fa2-676d-487d-9311-29d6e30afe45 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:11.719384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a1fa2-676d-487d-9311-29d6e30afe45 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:11.719563 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a1fa2-676d-487d-9311-29d6e30afe45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:11.739949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-b32a1fa2-676d-487d-9311-29d6e30afe45 None None] MFA Rules not processed for user `4c26b531c07a498c9617c9ee2f7bf7e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:11.741235 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-b32a1fa2-676d-487d-9311-29d6e30afe45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:11.820301 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c71ea-ae04-41bb-ba86-29d41fa6fca6 service cinder] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:11.825151 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f30c71ea-ae04-41bb-ba86-29d41fa6fca6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:11.844573 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1298/2604] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:11.845695 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a1fa2-676d-487d-9311-29d6e30afe45 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:11.847121 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-b32a1fa2-676d-487d-9311-29d6e30afe45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:11.870119 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a1fa2-676d-487d-9311-29d6e30afe45 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:11.874698 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42887c37-5991-450b-ac5e-c4d4f44f9674 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:11.875807 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42887c37-5991-450b-ac5e-c4d4f44f9674 service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:11.876139 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42887c37-5991-450b-ac5e-c4d4f44f9674 service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:11.876363 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42887c37-5991-450b-ac5e-c4d4f44f9674 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:11.895547 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42887c37-5991-450b-ac5e-c4d4f44f9674 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:11.896893 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42887c37-5991-450b-ac5e-c4d4f44f9674 service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:11.908399 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1299/2605] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:21:11 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:21:11.917815 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c71ea-ae04-41bb-ba86-29d41fa6fca6 service cinder] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:11.924092 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d3827d-b9e5-4ab9-a08e-055cacd4cec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:11.924779 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3827d-b9e5-4ab9-a08e-055cacd4cec7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:11.925063 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3827d-b9e5-4ab9-a08e-055cacd4cec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:11.925336 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3827d-b9e5-4ab9-a08e-055cacd4cec7 admin admin] PATH_INFO: `/v3/users/0cafcc70e0e74a5fa7c5ce530e0e848f` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:11.926479 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d3827d-b9e5-4ab9-a08e-055cacd4cec7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cafcc70e0e74a5fa7c5ce530e0e848f)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:11.928442 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d3827d-b9e5-4ab9-a08e-055cacd4cec7 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:11.996485 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-36d3827d-b9e5-4ab9-a08e-055cacd4cec7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cafcc70e0e74a5fa7c5ce530e0e848f'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:12.007965 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-36d3827d-b9e5-4ab9-a08e-055cacd4cec7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cafcc70e0e74a5fa7c5ce530e0e848f'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:12.009049 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-36d3827d-b9e5-4ab9-a08e-055cacd4cec7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cafcc70e0e74a5fa7c5ce530e0e848f'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:12.009940 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-36d3827d-b9e5-4ab9-a08e-055cacd4cec7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cafcc70e0e74a5fa7c5ce530e0e848f'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:12.019590 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-36d3827d-b9e5-4ab9-a08e-055cacd4cec7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cafcc70e0e74a5fa7c5ce530e0e848f'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:12.058056 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1307/2606] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:21:11 2022] DELETE /identity/v3/users/0cafcc70e0e74a5fa7c5ce530e0e848f => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:12.260123 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42887c37-5991-450b-ac5e-c4d4f44f9674 service cinder] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:12.275313 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42887c37-5991-450b-ac5e-c4d4f44f9674 service cinder] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:12.293751 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:12.348919 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34be154-1bf1-4e1a-8193-3031d59728aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:12.349470 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34be154-1bf1-4e1a-8193-3031d59728aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:12.349927 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34be154-1bf1-4e1a-8193-3031d59728aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:12.350304 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34be154-1bf1-4e1a-8193-3031d59728aa admin admin] PATH_INFO: `/v3/projects/701f62e5a7b14dfc86f2dcc5aa39d958` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:12.351686 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34be154-1bf1-4e1a-8193-3031d59728aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=701f62e5a7b14dfc86f2dcc5aa39d958)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:12.353941 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34be154-1bf1-4e1a-8193-3031d59728aa admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:12.405468 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-d34be154-1bf1-4e1a-8193-3031d59728aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '701f62e5a7b14dfc86f2dcc5aa39d958', 'request_id': 'req-d34be154-1bf1-4e1a-8193-3031d59728aa'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:12.411301 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-d34be154-1bf1-4e1a-8193-3031d59728aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '701f62e5a7b14dfc86f2dcc5aa39d958', 'request_id': 'req-d34be154-1bf1-4e1a-8193-3031d59728aa'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:12.420979 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-d34be154-1bf1-4e1a-8193-3031d59728aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '701f62e5a7b14dfc86f2dcc5aa39d958', 'request_id': 'req-d34be154-1bf1-4e1a-8193-3031d59728aa'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:12.433868 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-d34be154-1bf1-4e1a-8193-3031d59728aa admin admin] The token cache is being invalidate because project 701f62e5a7b14dfc86f2dcc5aa39d958 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:21:12.434165 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-d34be154-1bf1-4e1a-8193-3031d59728aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:12.434900 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-d34be154-1bf1-4e1a-8193-3031d59728aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:12.436694 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1300/2607] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:21:12 2022] DELETE /identity/v3/projects/701f62e5a7b14dfc86f2dcc5aa39d958 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:12.441640 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8d92b-981d-445a-97b3-433bddea7a72 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:12.441933 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8d92b-981d-445a-97b3-433bddea7a72 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:12.442201 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8d92b-981d-445a-97b3-433bddea7a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:12.461856 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-8af8d92b-981d-445a-97b3-433bddea7a72 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:12.463343 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-8af8d92b-981d-445a-97b3-433bddea7a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:12.474444 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:12.529909 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1308/2608] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:12.538643 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6acbc-63fd-4cb2-91df-f227551045e5 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:12.538959 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6acbc-63fd-4cb2-91df-f227551045e5 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:12.539246 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6acbc-63fd-4cb2-91df-f227551045e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:12.562039 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-54f6acbc-63fd-4cb2-91df-f227551045e5 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:12.564024 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-54f6acbc-63fd-4cb2-91df-f227551045e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:12.572677 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1301/2609] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:12.577121 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8d92b-981d-445a-97b3-433bddea7a72 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:12.603396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8d92b-981d-445a-97b3-433bddea7a72 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:12.609289 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e0481-5d11-468e-b5ae-6e3fbb8b8d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:12.609912 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e0481-5d11-468e-b5ae-6e3fbb8b8d18 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:12.610191 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e0481-5d11-468e-b5ae-6e3fbb8b8d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:12.610464 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e0481-5d11-468e-b5ae-6e3fbb8b8d18 admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:12.611158 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e0481-5d11-468e-b5ae-6e3fbb8b8d18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:21:12.611459 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e0481-5d11-468e-b5ae-6e3fbb8b8d18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:12.613056 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e0481-5d11-468e-b5ae-6e3fbb8b8d18 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:12.630885 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1309/2610] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:21:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:21:12.634703 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6acbc-63fd-4cb2-91df-f227551045e5 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:12.647170 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6acbc-63fd-4cb2-91df-f227551045e5 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:12.652285 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43622876-dec3-43ef-883d-897eb358f04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:12.652901 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43622876-dec3-43ef-883d-897eb358f04e admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:12.653183 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43622876-dec3-43ef-883d-897eb358f04e admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:12.654382 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43622876-dec3-43ef-883d-897eb358f04e admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:12.655800 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43622876-dec3-43ef-883d-897eb358f04e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:12.658451 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43622876-dec3-43ef-883d-897eb358f04e admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:12.692064 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1302/2611] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:21:12 2022] POST /identity/v3/projects => generated 405 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:21:12.696328 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e0481-5d11-468e-b5ae-6e3fbb8b8d18 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:12.722568 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e0481-5d11-468e-b5ae-6e3fbb8b8d18 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:12.785043 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227da156-5780-4141-9d73-0f7f5ea6dbed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:12.785651 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227da156-5780-4141-9d73-0f7f5ea6dbed admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:12.785930 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227da156-5780-4141-9d73-0f7f5ea6dbed admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:12.786216 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227da156-5780-4141-9d73-0f7f5ea6dbed admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:12.786830 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227da156-5780-4141-9d73-0f7f5ea6dbed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:12.788743 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227da156-5780-4141-9d73-0f7f5ea6dbed admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:12.809085 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1310/2612] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:21:12 2022] POST /identity/v3/users => generated 303 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:21:12.813647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43622876-dec3-43ef-883d-897eb358f04e admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:12.824521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43622876-dec3-43ef-883d-897eb358f04e admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:12.829914 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b874d-00c2-44b1-bc9b-b3dfcf754a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:12.830466 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b874d-00c2-44b1-bc9b-b3dfcf754a5b admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:12.830779 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b874d-00c2-44b1-bc9b-b3dfcf754a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:12.831078 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b874d-00c2-44b1-bc9b-b3dfcf754a5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:12.831721 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2b874d-00c2-44b1-bc9b-b3dfcf754a5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:12.833322 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2b874d-00c2-44b1-bc9b-b3dfcf754a5b admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:12.845045 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1303/2613] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:21:12.849082 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227da156-5780-4141-9d73-0f7f5ea6dbed admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:12.859157 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227da156-5780-4141-9d73-0f7f5ea6dbed admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:12.863934 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d343d90-3f58-421d-b24a-389274a68f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:12.864468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d343d90-3f58-421d-b24a-389274a68f30 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:12.864754 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d343d90-3f58-421d-b24a-389274a68f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:12.865012 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d343d90-3f58-421d-b24a-389274a68f30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:12.865654 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d343d90-3f58-421d-b24a-389274a68f30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:12.867279 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d343d90-3f58-421d-b24a-389274a68f30 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:12.878649 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1311/2614] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:21:12.882986 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b874d-00c2-44b1-bc9b-b3dfcf754a5b admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:12.893396 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b874d-00c2-44b1-bc9b-b3dfcf754a5b admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:12.897988 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d5c8a-6840-4557-870a-0ba241bee9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:12.898578 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d5c8a-6840-4557-870a-0ba241bee9a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:12.898881 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d5c8a-6840-4557-870a-0ba241bee9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:12.899154 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d5c8a-6840-4557-870a-0ba241bee9a3 admin admin] PATH_INFO: `/v3/projects/16f4b8ab396d4e008bf4e9e903d5da7a/users/d964637e460a4714a590faa16e3a3ce4/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:12.914565 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1d5c8a-6840-4557-870a-0ba241bee9a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16f4b8ab396d4e008bf4e9e903d5da7a, user_id=d964637e460a4714a590faa16e3a3ce4, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:12.917239 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1d5c8a-6840-4557-870a-0ba241bee9a3 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:12.930366 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1304/2615] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:21:12 2022] PUT /identity/v3/projects/16f4b8ab396d4e008bf4e9e903d5da7a/users/d964637e460a4714a590faa16e3a3ce4/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:12.936048 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e741da26-f730-4bec-9be3-9d413fffcd1b None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:12.936345 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e741da26-f730-4bec-9be3-9d413fffcd1b None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:12.936626 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e741da26-f730-4bec-9be3-9d413fffcd1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:12.956213 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-e741da26-f730-4bec-9be3-9d413fffcd1b None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:12.957435 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e741da26-f730-4bec-9be3-9d413fffcd1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:13.019143 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1312/2616] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:13.028673 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d5c8a-6840-4557-870a-0ba241bee9a3 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:13.030469 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-6d1d5c8a-6840-4557-870a-0ba241bee9a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:13.074729 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:13.092364 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d5c8a-6840-4557-870a-0ba241bee9a3 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:13.095970 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d5661-10bd-4cb2-803f-a9d8febc8799 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:13.096622 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d5661-10bd-4cb2-803f-a9d8febc8799 service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:13.096897 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d5661-10bd-4cb2-803f-a9d8febc8799 service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:13.097163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d5661-10bd-4cb2-803f-a9d8febc8799 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:13.106131 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d5661-10bd-4cb2-803f-a9d8febc8799 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:13.107619 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d5661-10bd-4cb2-803f-a9d8febc8799 service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:13.117512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1305/2617] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:21:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:21:13.972142 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd1bfc4-0fee-4dfd-9327-2a0b2fe39735 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:13.973184 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd1bfc4-0fee-4dfd-9327-2a0b2fe39735 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:13.973484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd1bfc4-0fee-4dfd-9327-2a0b2fe39735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:14.012777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:14.016358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-efd1bfc4-0fee-4dfd-9327-2a0b2fe39735 None None] MFA Rules not processed for user `d964637e460a4714a590faa16e3a3ce4`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:14.017696 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-efd1bfc4-0fee-4dfd-9327-2a0b2fe39735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:14.119783 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1313/2618] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:13 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:14.124677 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d5661-10bd-4cb2-803f-a9d8febc8799 service neutron] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:14.185623 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d5661-10bd-4cb2-803f-a9d8febc8799 service neutron] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:14.190465 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6ddad-e15d-4178-81e8-8331b7bf8a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:14.191132 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6ddad-e15d-4178-81e8-8331b7bf8a23 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:14.191411 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6ddad-e15d-4178-81e8-8331b7bf8a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:14.191682 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6ddad-e15d-4178-81e8-8331b7bf8a23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:14.192310 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb6ddad-e15d-4178-81e8-8331b7bf8a23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:14.194213 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb6ddad-e15d-4178-81e8-8331b7bf8a23 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:14.235059 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1306/2619] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:21:14 2022] POST /identity/v3/projects => generated 401 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:21:14.239301 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd1bfc4-0fee-4dfd-9327-2a0b2fe39735 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:14.250468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd1bfc4-0fee-4dfd-9327-2a0b2fe39735 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:14.307531 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610b453-0ebc-4b52-a593-f208d9ead331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:14.308136 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610b453-0ebc-4b52-a593-f208d9ead331 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:14.308465 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610b453-0ebc-4b52-a593-f208d9ead331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:14.308739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610b453-0ebc-4b52-a593-f208d9ead331 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:14.309375 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610b453-0ebc-4b52-a593-f208d9ead331 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:14.311595 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610b453-0ebc-4b52-a593-f208d9ead331 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:14.334578 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1314/2620] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:21:14 2022] POST /identity/v3/users => generated 307 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:21:14.338720 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6ddad-e15d-4178-81e8-8331b7bf8a23 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:14.350575 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6ddad-e15d-4178-81e8-8331b7bf8a23 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:14.355152 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07d923-f107-42ee-8a1f-93e148668636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:14.355753 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07d923-f107-42ee-8a1f-93e148668636 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:14.356027 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07d923-f107-42ee-8a1f-93e148668636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:14.356295 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07d923-f107-42ee-8a1f-93e148668636 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:14.356964 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07d923-f107-42ee-8a1f-93e148668636 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:14.358567 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07d923-f107-42ee-8a1f-93e148668636 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:14.372015 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1307/2621] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:21:14.373393 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610b453-0ebc-4b52-a593-f208d9ead331 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:14.383763 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610b453-0ebc-4b52-a593-f208d9ead331 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:14.389024 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e627e0-0af9-4f6c-a3d2-c39263f8e53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:14.389673 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e627e0-0af9-4f6c-a3d2-c39263f8e53f admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:14.389952 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e627e0-0af9-4f6c-a3d2-c39263f8e53f admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:14.390407 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e627e0-0af9-4f6c-a3d2-c39263f8e53f admin admin] PATH_INFO: `/v3/projects/4a8942946042464383f7e61600848a7a/users/e0c7aef7dfca47f3a85192cee096e9d6/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:14.406305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e627e0-0af9-4f6c-a3d2-c39263f8e53f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a8942946042464383f7e61600848a7a, user_id=e0c7aef7dfca47f3a85192cee096e9d6, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:14.408959 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e627e0-0af9-4f6c-a3d2-c39263f8e53f admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:14.423036 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1315/2622] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:21:14 2022] PUT /identity/v3/projects/4a8942946042464383f7e61600848a7a/users/e0c7aef7dfca47f3a85192cee096e9d6/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:21:14.426899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07d923-f107-42ee-8a1f-93e148668636 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:14.437549 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07d923-f107-42ee-8a1f-93e148668636 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:14.496812 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2299bbbb-608f-4b7c-8edb-818802961f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:14.497954 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2299bbbb-608f-4b7c-8edb-818802961f46 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:14.498228 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2299bbbb-608f-4b7c-8edb-818802961f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:14.498495 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2299bbbb-608f-4b7c-8edb-818802961f46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:14.499176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2299bbbb-608f-4b7c-8edb-818802961f46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:14.500752 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2299bbbb-608f-4b7c-8edb-818802961f46 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:14.511082 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1308/2623] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:21:14.516059 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e627e0-0af9-4f6c-a3d2-c39263f8e53f admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:14.526078 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e627e0-0af9-4f6c-a3d2-c39263f8e53f admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:14.531409 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63392e92-545e-4bc0-a2be-486d662fc19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:14.531613 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63392e92-545e-4bc0-a2be-486d662fc19b admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:14.531943 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63392e92-545e-4bc0-a2be-486d662fc19b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:14.532086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63392e92-545e-4bc0-a2be-486d662fc19b admin admin] PATH_INFO: `/v3/projects/4a8942946042464383f7e61600848a7a/users/e0c7aef7dfca47f3a85192cee096e9d6/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:14.534345 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63392e92-545e-4bc0-a2be-486d662fc19b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a8942946042464383f7e61600848a7a, user_id=e0c7aef7dfca47f3a85192cee096e9d6, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:14.537058 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63392e92-545e-4bc0-a2be-486d662fc19b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:14.551196 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1316/2624] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:21:14 2022] PUT /identity/v3/projects/4a8942946042464383f7e61600848a7a/users/e0c7aef7dfca47f3a85192cee096e9d6/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:15.354341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736feaf-7a6f-44fc-8d72-9df0cac30915 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:15.354891 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736feaf-7a6f-44fc-8d72-9df0cac30915 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:15.355038 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736feaf-7a6f-44fc-8d72-9df0cac30915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:15.385622 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-7736feaf-7a6f-44fc-8d72-9df0cac30915 None None] MFA Rules not processed for user `e0c7aef7dfca47f3a85192cee096e9d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:15.387120 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-7736feaf-7a6f-44fc-8d72-9df0cac30915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:15.434820 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:15.468346 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1309/2625] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:15 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:16.544726 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63392e92-545e-4bc0-a2be-486d662fc19b admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:16.592817 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:16.614990 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63392e92-545e-4bc0-a2be-486d662fc19b admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:16.619832 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b41fa3-e9b0-43f9-a5bb-0e9114851e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:16.620639 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b41fa3-e9b0-43f9-a5bb-0e9114851e80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:16.620929 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b41fa3-e9b0-43f9-a5bb-0e9114851e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:16.621197 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b41fa3-e9b0-43f9-a5bb-0e9114851e80 admin admin] PATH_INFO: `/v3/users/d964637e460a4714a590faa16e3a3ce4` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:16.623227 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b41fa3-e9b0-43f9-a5bb-0e9114851e80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d964637e460a4714a590faa16e3a3ce4)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:16.625266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b41fa3-e9b0-43f9-a5bb-0e9114851e80 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:16.674117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-27b41fa3-e9b0-43f9-a5bb-0e9114851e80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd964637e460a4714a590faa16e3a3ce4'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:16.682107 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-27b41fa3-e9b0-43f9-a5bb-0e9114851e80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd964637e460a4714a590faa16e3a3ce4'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:16.682797 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-27b41fa3-e9b0-43f9-a5bb-0e9114851e80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd964637e460a4714a590faa16e3a3ce4'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:16.686225 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-27b41fa3-e9b0-43f9-a5bb-0e9114851e80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd964637e460a4714a590faa16e3a3ce4'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:16.693888 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-27b41fa3-e9b0-43f9-a5bb-0e9114851e80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd964637e460a4714a590faa16e3a3ce4'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:16.712779 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1317/2626] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:21:16 2022] DELETE /identity/v3/users/d964637e460a4714a590faa16e3a3ce4 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:17.177248 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736feaf-7a6f-44fc-8d72-9df0cac30915 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:17.201967 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736feaf-7a6f-44fc-8d72-9df0cac30915 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:17.308734 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b9049-4846-4dd6-b6ef-6635e105827b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:17.309530 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b9049-4846-4dd6-b6ef-6635e105827b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:17.310791 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b9049-4846-4dd6-b6ef-6635e105827b admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:17.311180 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b9049-4846-4dd6-b6ef-6635e105827b admin admin] PATH_INFO: `/v3/projects/16f4b8ab396d4e008bf4e9e903d5da7a` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:17.312860 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b9049-4846-4dd6-b6ef-6635e105827b admin admin] RBAC: Authorizing `identity:delete_project(project_id=16f4b8ab396d4e008bf4e9e903d5da7a)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:17.315195 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b9049-4846-4dd6-b6ef-6635e105827b admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:17.376621 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-2f6b9049-4846-4dd6-b6ef-6635e105827b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16f4b8ab396d4e008bf4e9e903d5da7a', 'request_id': 'req-2f6b9049-4846-4dd6-b6ef-6635e105827b'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:17.382522 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-2f6b9049-4846-4dd6-b6ef-6635e105827b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16f4b8ab396d4e008bf4e9e903d5da7a', 'request_id': 'req-2f6b9049-4846-4dd6-b6ef-6635e105827b'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:17.391509 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-2f6b9049-4846-4dd6-b6ef-6635e105827b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16f4b8ab396d4e008bf4e9e903d5da7a', 'request_id': 'req-2f6b9049-4846-4dd6-b6ef-6635e105827b'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:17.403171 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-2f6b9049-4846-4dd6-b6ef-6635e105827b admin admin] The token cache is being invalidate because project 16f4b8ab396d4e008bf4e9e903d5da7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:21:17.403965 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-2f6b9049-4846-4dd6-b6ef-6635e105827b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:17.404606 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-2f6b9049-4846-4dd6-b6ef-6635e105827b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:17.406274 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1310/2627] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:21:17 2022] DELETE /identity/v3/projects/16f4b8ab396d4e008bf4e9e903d5da7a => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:17.410060 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b41fa3-e9b0-43f9-a5bb-0e9114851e80 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:17.411581 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-27b41fa3-e9b0-43f9-a5bb-0e9114851e80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:17.425179 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b41fa3-e9b0-43f9-a5bb-0e9114851e80 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:17.486849 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb326d64-d410-451c-9f46-9472d60b4166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:17.487624 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb326d64-d410-451c-9f46-9472d60b4166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:17.487899 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb326d64-d410-451c-9f46-9472d60b4166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:17.488167 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb326d64-d410-451c-9f46-9472d60b4166 admin admin] PATH_INFO: `/v3/users/e0c7aef7dfca47f3a85192cee096e9d6` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:17.489318 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb326d64-d410-451c-9f46-9472d60b4166 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0c7aef7dfca47f3a85192cee096e9d6)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:17.491312 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb326d64-d410-451c-9f46-9472d60b4166 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:17.544158 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-eb326d64-d410-451c-9f46-9472d60b4166 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0c7aef7dfca47f3a85192cee096e9d6'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:17.551729 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-eb326d64-d410-451c-9f46-9472d60b4166 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0c7aef7dfca47f3a85192cee096e9d6'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:17.552428 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-eb326d64-d410-451c-9f46-9472d60b4166 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0c7aef7dfca47f3a85192cee096e9d6'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:17.553027 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-eb326d64-d410-451c-9f46-9472d60b4166 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0c7aef7dfca47f3a85192cee096e9d6'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:17.557441 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:17.561186 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-eb326d64-d410-451c-9f46-9472d60b4166 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0c7aef7dfca47f3a85192cee096e9d6'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:17.579931 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1318/2628] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:21:17 2022] DELETE /identity/v3/users/e0c7aef7dfca47f3a85192cee096e9d6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:17.755586 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b9049-4846-4dd6-b6ef-6635e105827b admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:17.757071 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-2f6b9049-4846-4dd6-b6ef-6635e105827b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:17.776130 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b9049-4846-4dd6-b6ef-6635e105827b admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:17.834681 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb878153-245d-47ef-b68a-42e8af13a64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:17.835290 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb878153-245d-47ef-b68a-42e8af13a64e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:17.835658 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb878153-245d-47ef-b68a-42e8af13a64e admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:17.835828 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb878153-245d-47ef-b68a-42e8af13a64e admin admin] PATH_INFO: `/v3/projects/4a8942946042464383f7e61600848a7a` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:17.836948 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb878153-245d-47ef-b68a-42e8af13a64e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a8942946042464383f7e61600848a7a)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:17.838832 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb878153-245d-47ef-b68a-42e8af13a64e admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:17.849543 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:17.886471 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-eb878153-245d-47ef-b68a-42e8af13a64e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a8942946042464383f7e61600848a7a', 'request_id': 'req-eb878153-245d-47ef-b68a-42e8af13a64e'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:17.892613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-eb878153-245d-47ef-b68a-42e8af13a64e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a8942946042464383f7e61600848a7a', 'request_id': 'req-eb878153-245d-47ef-b68a-42e8af13a64e'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:17.899816 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-eb878153-245d-47ef-b68a-42e8af13a64e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a8942946042464383f7e61600848a7a', 'request_id': 'req-eb878153-245d-47ef-b68a-42e8af13a64e'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:17.911386 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-eb878153-245d-47ef-b68a-42e8af13a64e admin admin] The token cache is being invalidate because project 4a8942946042464383f7e61600848a7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:21:17.911773 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-eb878153-245d-47ef-b68a-42e8af13a64e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:17.912398 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-eb878153-245d-47ef-b68a-42e8af13a64e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:17.913864 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1311/2629] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:21:17 2022] DELETE /identity/v3/projects/4a8942946042464383f7e61600848a7a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:17.919404 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f851b9-1aa2-4f64-bdc1-4de9db8e1fb6 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:17.919690 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f851b9-1aa2-4f64-bdc1-4de9db8e1fb6 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:17.919992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f851b9-1aa2-4f64-bdc1-4de9db8e1fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:17.976051 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-47f851b9-1aa2-4f64-bdc1-4de9db8e1fb6 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:17.977549 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-47f851b9-1aa2-4f64-bdc1-4de9db8e1fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:18.034649 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1319/2630] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:21:18.042322 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbe9d3-04ef-4f72-8f77-028a9774d898 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:18.042603 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbe9d3-04ef-4f72-8f77-028a9774d898 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:18.042899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbe9d3-04ef-4f72-8f77-028a9774d898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:18.063899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-47dbe9d3-04ef-4f72-8f77-028a9774d898 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:18.065098 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-47dbe9d3-04ef-4f72-8f77-028a9774d898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:18.072978 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1312/2631] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:18.077191 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f851b9-1aa2-4f64-bdc1-4de9db8e1fb6 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:18.086788 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f851b9-1aa2-4f64-bdc1-4de9db8e1fb6 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:18.091267 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d58dd-55ea-4406-a69e-229f7125d26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:18.091854 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d58dd-55ea-4406-a69e-229f7125d26f admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:18.092136 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d58dd-55ea-4406-a69e-229f7125d26f admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:18.092412 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d58dd-55ea-4406-a69e-229f7125d26f admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:18.093056 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d58dd-55ea-4406-a69e-229f7125d26f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:21:18.093356 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d58dd-55ea-4406-a69e-229f7125d26f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:18.094947 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d58dd-55ea-4406-a69e-229f7125d26f admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:18.106689 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1320/2632] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:21:18.110652 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dbe9d3-04ef-4f72-8f77-028a9774d898 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:18.119810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dbe9d3-04ef-4f72-8f77-028a9774d898 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:18.125245 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09951f5a-aa21-4bfe-b68e-637a1529fbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:18.125860 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09951f5a-aa21-4bfe-b68e-637a1529fbba admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:18.126137 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09951f5a-aa21-4bfe-b68e-637a1529fbba admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:18.126408 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09951f5a-aa21-4bfe-b68e-637a1529fbba admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:18.127072 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09951f5a-aa21-4bfe-b68e-637a1529fbba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:18.128940 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09951f5a-aa21-4bfe-b68e-637a1529fbba admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:18.156131 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1313/2633] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:21:18 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:21:18.160219 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d58dd-55ea-4406-a69e-229f7125d26f admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:18.170649 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d58dd-55ea-4406-a69e-229f7125d26f admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:18.225060 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52162f-c123-42b1-ab58-c632911d1381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:18.225654 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52162f-c123-42b1-ab58-c632911d1381 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:18.225935 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52162f-c123-42b1-ab58-c632911d1381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:18.226208 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52162f-c123-42b1-ab58-c632911d1381 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:18.226867 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e52162f-c123-42b1-ab58-c632911d1381 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:18.228742 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e52162f-c123-42b1-ab58-c632911d1381 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:18.250561 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1321/2634] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:21:18 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:21:18.254716 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09951f5a-aa21-4bfe-b68e-637a1529fbba admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:18.264875 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09951f5a-aa21-4bfe-b68e-637a1529fbba admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:18.269787 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa8d1a-f021-4440-bcf8-01bc68697af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:18.270384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa8d1a-f021-4440-bcf8-01bc68697af0 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:18.270667 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa8d1a-f021-4440-bcf8-01bc68697af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:18.270974 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa8d1a-f021-4440-bcf8-01bc68697af0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:18.271618 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aa8d1a-f021-4440-bcf8-01bc68697af0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:18.273308 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aa8d1a-f021-4440-bcf8-01bc68697af0 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:18.284115 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1314/2635] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:21:18.287962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52162f-c123-42b1-ab58-c632911d1381 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:18.298519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52162f-c123-42b1-ab58-c632911d1381 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:18.303159 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca82c63-004a-4c1c-80a0-0fb0e03167d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:18.303750 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca82c63-004a-4c1c-80a0-0fb0e03167d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:18.304030 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca82c63-004a-4c1c-80a0-0fb0e03167d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:18.304304 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca82c63-004a-4c1c-80a0-0fb0e03167d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:18.304943 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca82c63-004a-4c1c-80a0-0fb0e03167d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:18.306508 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca82c63-004a-4c1c-80a0-0fb0e03167d5 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:18.317253 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1322/2636] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:21:18.321364 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa8d1a-f021-4440-bcf8-01bc68697af0 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:18.331376 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa8d1a-f021-4440-bcf8-01bc68697af0 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:18.335938 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d6653-243a-47f2-a5b2-dc7142babe90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:18.336577 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d6653-243a-47f2-a5b2-dc7142babe90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:18.336920 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d6653-243a-47f2-a5b2-dc7142babe90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:18.337125 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d6653-243a-47f2-a5b2-dc7142babe90 admin admin] PATH_INFO: `/v3/projects/a1a514e4885b48eeb2c52da39cf543a6/users/3caa9f0d7b7e4e1091412bae334dac2b/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:18.352619 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441d6653-243a-47f2-a5b2-dc7142babe90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1a514e4885b48eeb2c52da39cf543a6, user_id=3caa9f0d7b7e4e1091412bae334dac2b, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:18.355242 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441d6653-243a-47f2-a5b2-dc7142babe90 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:18.369071 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1315/2637] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:21:18 2022] PUT /identity/v3/projects/a1a514e4885b48eeb2c52da39cf543a6/users/3caa9f0d7b7e4e1091412bae334dac2b/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:18.380922 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ab579-b695-474f-9f44-a77cbfdd76da None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:18.381276 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ab579-b695-474f-9f44-a77cbfdd76da None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:18.381604 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ab579-b695-474f-9f44-a77cbfdd76da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:18.402623 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:18.411517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-ee7ab579-b695-474f-9f44-a77cbfdd76da None None] MFA Rules not processed for user `3caa9f0d7b7e4e1091412bae334dac2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:18.413209 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ee7ab579-b695-474f-9f44-a77cbfdd76da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:18.484358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1323/2638] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:18.488859 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d6653-243a-47f2-a5b2-dc7142babe90 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:18.552556 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d6653-243a-47f2-a5b2-dc7142babe90 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:18.557814 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889017e3-709a-4645-a84e-b08c2a82912f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:18.558414 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889017e3-709a-4645-a84e-b08c2a82912f admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:18.558687 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889017e3-709a-4645-a84e-b08c2a82912f admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:18.558984 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889017e3-709a-4645-a84e-b08c2a82912f admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:18.559650 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889017e3-709a-4645-a84e-b08c2a82912f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:18.561777 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889017e3-709a-4645-a84e-b08c2a82912f admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:18.588770 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1316/2639] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:21:18 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:21:18.593087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7ab579-b695-474f-9f44-a77cbfdd76da None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:18.604471 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7ab579-b695-474f-9f44-a77cbfdd76da None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:18.669310 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f658d36e-b234-4fad-89da-3b64e8f22957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:18.670523 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658d36e-b234-4fad-89da-3b64e8f22957 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:18.670809 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658d36e-b234-4fad-89da-3b64e8f22957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:18.671117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658d36e-b234-4fad-89da-3b64e8f22957 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:18.671875 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658d36e-b234-4fad-89da-3b64e8f22957 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:18.674019 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658d36e-b234-4fad-89da-3b64e8f22957 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:18.704461 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1324/2640] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:21:18 2022] POST /identity/v3/users => generated 300 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:21:18.709309 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889017e3-709a-4645-a84e-b08c2a82912f admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:18.723287 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889017e3-709a-4645-a84e-b08c2a82912f admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:18.731162 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635a2c5-0974-4066-b8bb-782b8628c530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:18.732197 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635a2c5-0974-4066-b8bb-782b8628c530 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:18.732686 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635a2c5-0974-4066-b8bb-782b8628c530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:18.733179 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635a2c5-0974-4066-b8bb-782b8628c530 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:18.734378 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3635a2c5-0974-4066-b8bb-782b8628c530 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:18.737608 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3635a2c5-0974-4066-b8bb-782b8628c530 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:18.758905 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1317/2641] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:21:18.766404 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f658d36e-b234-4fad-89da-3b64e8f22957 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:18.785453 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f658d36e-b234-4fad-89da-3b64e8f22957 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:18.796763 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe1af86-0ca4-4d7c-80ce-dc46922325a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:18.797960 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1af86-0ca4-4d7c-80ce-dc46922325a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:18.798454 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1af86-0ca4-4d7c-80ce-dc46922325a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:18.798977 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1af86-0ca4-4d7c-80ce-dc46922325a2 admin admin] PATH_INFO: `/v3/projects/947513e813484ff0a3a4fed0b939c451/users/f18544b541db4f74a24bc827e6f90d2a/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:18.837973 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe1af86-0ca4-4d7c-80ce-dc46922325a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=947513e813484ff0a3a4fed0b939c451, user_id=f18544b541db4f74a24bc827e6f90d2a, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:18.847239 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe1af86-0ca4-4d7c-80ce-dc46922325a2 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:18.875079 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1325/2642] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:21:18 2022] PUT /identity/v3/projects/947513e813484ff0a3a4fed0b939c451/users/f18544b541db4f74a24bc827e6f90d2a/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:18.880786 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635a2c5-0974-4066-b8bb-782b8628c530 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:18.897638 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635a2c5-0974-4066-b8bb-782b8628c530 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:18.906168 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:18.969929 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ed56f-c35d-4431-9997-3bfe5f722f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:18.970541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ed56f-c35d-4431-9997-3bfe5f722f6a admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:18.970827 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ed56f-c35d-4431-9997-3bfe5f722f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:18.971124 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ed56f-c35d-4431-9997-3bfe5f722f6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:18.971772 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ed56f-c35d-4431-9997-3bfe5f722f6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:18.973347 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ed56f-c35d-4431-9997-3bfe5f722f6a admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:18.983373 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1318/2643] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:21:18.987646 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe1af86-0ca4-4d7c-80ce-dc46922325a2 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:18.998265 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe1af86-0ca4-4d7c-80ce-dc46922325a2 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:19.002789 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72b40a-a2ff-4191-8128-10cfdccc9c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:19.003449 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72b40a-a2ff-4191-8128-10cfdccc9c4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:19.003795 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72b40a-a2ff-4191-8128-10cfdccc9c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:19.004060 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72b40a-a2ff-4191-8128-10cfdccc9c4a admin admin] PATH_INFO: `/v3/projects/947513e813484ff0a3a4fed0b939c451/users/f18544b541db4f74a24bc827e6f90d2a/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:19.006024 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b72b40a-a2ff-4191-8128-10cfdccc9c4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=947513e813484ff0a3a4fed0b939c451, user_id=f18544b541db4f74a24bc827e6f90d2a, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:19.008704 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b72b40a-a2ff-4191-8128-10cfdccc9c4a admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:19.019524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1326/2644] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:21:18 2022] PUT /identity/v3/projects/947513e813484ff0a3a4fed0b939c451/users/f18544b541db4f74a24bc827e6f90d2a/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:21:19.029404 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f7b55-eead-4530-8116-14c67b70cab5 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:19.029692 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f7b55-eead-4530-8116-14c67b70cab5 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:19.029996 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f7b55-eead-4530-8116-14c67b70cab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:19.058224 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-337f7b55-eead-4530-8116-14c67b70cab5 None None] MFA Rules not processed for user `f18544b541db4f74a24bc827e6f90d2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:19.059968 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-337f7b55-eead-4530-8116-14c67b70cab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:19.126777 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1319/2645] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:19 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:19.139656 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72b40a-a2ff-4191-8128-10cfdccc9c4a admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:19.141413 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-5b72b40a-a2ff-4191-8128-10cfdccc9c4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:19.203738 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:19.222864 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72b40a-a2ff-4191-8128-10cfdccc9c4a admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:19.226493 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03c2e52-643f-407d-aa58-6cbb0e4a81a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:19.227129 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c2e52-643f-407d-aa58-6cbb0e4a81a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:19.227413 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c2e52-643f-407d-aa58-6cbb0e4a81a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:19.227691 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c2e52-643f-407d-aa58-6cbb0e4a81a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:19.236777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03c2e52-643f-407d-aa58-6cbb0e4a81a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:19.238225 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03c2e52-643f-407d-aa58-6cbb0e4a81a7 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:19.248790 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1327/2646] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:21:19 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:21:19.280015 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f7b55-eead-4530-8116-14c67b70cab5 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:19.341823 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f7b55-eead-4530-8116-14c67b70cab5 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:19.346387 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9364d5-c640-470f-adc6-234729e5a1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:19.347037 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9364d5-c640-470f-adc6-234729e5a1ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:19.347311 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9364d5-c640-470f-adc6-234729e5a1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:19.347572 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9364d5-c640-470f-adc6-234729e5a1ed admin admin] PATH_INFO: `/v3/users/3caa9f0d7b7e4e1091412bae334dac2b` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:19.348712 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9364d5-c640-470f-adc6-234729e5a1ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=3caa9f0d7b7e4e1091412bae334dac2b)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:19.350619 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9364d5-c640-470f-adc6-234729e5a1ed admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:19.383941 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:19.397044 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-fe9364d5-c640-470f-adc6-234729e5a1ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3caa9f0d7b7e4e1091412bae334dac2b'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:19.403448 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-fe9364d5-c640-470f-adc6-234729e5a1ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3caa9f0d7b7e4e1091412bae334dac2b'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:19.404311 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-fe9364d5-c640-470f-adc6-234729e5a1ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3caa9f0d7b7e4e1091412bae334dac2b'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:19.404966 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-fe9364d5-c640-470f-adc6-234729e5a1ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3caa9f0d7b7e4e1091412bae334dac2b'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:19.411837 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-fe9364d5-c640-470f-adc6-234729e5a1ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3caa9f0d7b7e4e1091412bae334dac2b'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:19.429411 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1320/2647] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:21:19 2022] DELETE /identity/v3/users/3caa9f0d7b7e4e1091412bae334dac2b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:21:19.434113 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdbc27-5710-4488-b9c9-27bf78ef7702 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:19.434969 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdbc27-5710-4488-b9c9-27bf78ef7702 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:19.435495 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdbc27-5710-4488-b9c9-27bf78ef7702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:19.454097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-0dfdbc27-5710-4488-b9c9-27bf78ef7702 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:19.455336 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-0dfdbc27-5710-4488-b9c9-27bf78ef7702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:19.513978 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1328/2648] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:19.523715 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9364d5-c640-470f-adc6-234729e5a1ed admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:19.525141 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-fe9364d5-c640-470f-adc6-234729e5a1ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:19.581837 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9364d5-c640-470f-adc6-234729e5a1ed admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:19.585534 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc6905-a3bf-4c47-b4a7-1e8328421e7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:19.586145 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc6905-a3bf-4c47-b4a7-1e8328421e7c service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:19.586421 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc6905-a3bf-4c47-b4a7-1e8328421e7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:19.586690 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc6905-a3bf-4c47-b4a7-1e8328421e7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:19.595937 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc6905-a3bf-4c47-b4a7-1e8328421e7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:19.597353 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc6905-a3bf-4c47-b4a7-1e8328421e7c service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:19.607282 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1321/2649] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:21:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:21:19.874141 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfdbc27-5710-4488-b9c9-27bf78ef7702 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:19.875573 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-0dfdbc27-5710-4488-b9c9-27bf78ef7702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:19.889089 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfdbc27-5710-4488-b9c9-27bf78ef7702 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:19.894133 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d22b4c-9cec-46b2-bdfe-3fd7706d4f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:19.894861 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d22b4c-9cec-46b2-bdfe-3fd7706d4f62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:19.895245 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d22b4c-9cec-46b2-bdfe-3fd7706d4f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:19.895575 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d22b4c-9cec-46b2-bdfe-3fd7706d4f62 admin admin] PATH_INFO: `/v3/projects/a1a514e4885b48eeb2c52da39cf543a6` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:19.896821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d22b4c-9cec-46b2-bdfe-3fd7706d4f62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1a514e4885b48eeb2c52da39cf543a6)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:19.898773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d22b4c-9cec-46b2-bdfe-3fd7706d4f62 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:19.949128 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-67d22b4c-9cec-46b2-bdfe-3fd7706d4f62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1a514e4885b48eeb2c52da39cf543a6', 'request_id': 'req-67d22b4c-9cec-46b2-bdfe-3fd7706d4f62'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:19.957326 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-67d22b4c-9cec-46b2-bdfe-3fd7706d4f62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1a514e4885b48eeb2c52da39cf543a6', 'request_id': 'req-67d22b4c-9cec-46b2-bdfe-3fd7706d4f62'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:19.963190 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-67d22b4c-9cec-46b2-bdfe-3fd7706d4f62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1a514e4885b48eeb2c52da39cf543a6', 'request_id': 'req-67d22b4c-9cec-46b2-bdfe-3fd7706d4f62'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:19.976461 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-67d22b4c-9cec-46b2-bdfe-3fd7706d4f62 admin admin] The token cache is being invalidate because project a1a514e4885b48eeb2c52da39cf543a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:21:19.976751 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-67d22b4c-9cec-46b2-bdfe-3fd7706d4f62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:19.977392 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-67d22b4c-9cec-46b2-bdfe-3fd7706d4f62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:19.978885 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1329/2650] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:21:19 2022] DELETE /identity/v3/projects/a1a514e4885b48eeb2c52da39cf543a6 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:19.983611 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc6905-a3bf-4c47-b4a7-1e8328421e7c service neutron] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:19.995415 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc6905-a3bf-4c47-b4a7-1e8328421e7c service neutron] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:20.056898 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5555727-fa44-431e-8835-87673f3bb00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:20.057668 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5555727-fa44-431e-8835-87673f3bb00f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:20.058163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5555727-fa44-431e-8835-87673f3bb00f admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:20.058532 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5555727-fa44-431e-8835-87673f3bb00f admin admin] PATH_INFO: `/v3/users/f18544b541db4f74a24bc827e6f90d2a` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:20.059840 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5555727-fa44-431e-8835-87673f3bb00f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f18544b541db4f74a24bc827e6f90d2a)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:20.062118 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5555727-fa44-431e-8835-87673f3bb00f admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:20.113101 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-a5555727-fa44-431e-8835-87673f3bb00f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f18544b541db4f74a24bc827e6f90d2a'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:20.121558 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-a5555727-fa44-431e-8835-87673f3bb00f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f18544b541db4f74a24bc827e6f90d2a'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:20.122238 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-a5555727-fa44-431e-8835-87673f3bb00f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f18544b541db4f74a24bc827e6f90d2a'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:20.122895 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-a5555727-fa44-431e-8835-87673f3bb00f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f18544b541db4f74a24bc827e6f90d2a'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:20.124292 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:20.135213 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-a5555727-fa44-431e-8835-87673f3bb00f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f18544b541db4f74a24bc827e6f90d2a'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:20.154178 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1322/2651] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:21:19 2022] DELETE /identity/v3/users/f18544b541db4f74a24bc827e6f90d2a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:20.426114 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d22b4c-9cec-46b2-bdfe-3fd7706d4f62 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:20.427586 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-67d22b4c-9cec-46b2-bdfe-3fd7706d4f62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:20.441210 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d22b4c-9cec-46b2-bdfe-3fd7706d4f62 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:20.505305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d4f35-26cc-44f4-baa7-e3ce8a6fc6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:20.505926 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d4f35-26cc-44f4-baa7-e3ce8a6fc6dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:20.506202 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d4f35-26cc-44f4-baa7-e3ce8a6fc6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:20.506468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d4f35-26cc-44f4-baa7-e3ce8a6fc6dc admin admin] PATH_INFO: `/v3/projects/947513e813484ff0a3a4fed0b939c451` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:20.507652 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d4f35-26cc-44f4-baa7-e3ce8a6fc6dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=947513e813484ff0a3a4fed0b939c451)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:20.509596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d4f35-26cc-44f4-baa7-e3ce8a6fc6dc admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:20.516374 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:20.555970 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f04d4f35-26cc-44f4-baa7-e3ce8a6fc6dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '947513e813484ff0a3a4fed0b939c451', 'request_id': 'req-f04d4f35-26cc-44f4-baa7-e3ce8a6fc6dc'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:20.564230 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f04d4f35-26cc-44f4-baa7-e3ce8a6fc6dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '947513e813484ff0a3a4fed0b939c451', 'request_id': 'req-f04d4f35-26cc-44f4-baa7-e3ce8a6fc6dc'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:20.570280 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f04d4f35-26cc-44f4-baa7-e3ce8a6fc6dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '947513e813484ff0a3a4fed0b939c451', 'request_id': 'req-f04d4f35-26cc-44f4-baa7-e3ce8a6fc6dc'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:20.581547 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f04d4f35-26cc-44f4-baa7-e3ce8a6fc6dc admin admin] The token cache is being invalidate because project 947513e813484ff0a3a4fed0b939c451 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:21:20.581829 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f04d4f35-26cc-44f4-baa7-e3ce8a6fc6dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:20.582443 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f04d4f35-26cc-44f4-baa7-e3ce8a6fc6dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:20.583926 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1330/2652] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:21:20 2022] DELETE /identity/v3/projects/947513e813484ff0a3a4fed0b939c451 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:20.589065 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387198a9-d921-49d9-98be-eb657849c858 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:20.589494 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387198a9-d921-49d9-98be-eb657849c858 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:20.589676 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387198a9-d921-49d9-98be-eb657849c858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:20.609278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-387198a9-d921-49d9-98be-eb657849c858 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:20.610468 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-387198a9-d921-49d9-98be-eb657849c858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:20.672521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1323/2653] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:21:20.680705 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77bcda-9134-4080-b862-3cfd9536bf52 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:20.681064 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77bcda-9134-4080-b862-3cfd9536bf52 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:20.681392 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77bcda-9134-4080-b862-3cfd9536bf52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:20.703033 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-9e77bcda-9134-4080-b862-3cfd9536bf52 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:20.704520 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-9e77bcda-9134-4080-b862-3cfd9536bf52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:20.712538 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1331/2654] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:20.716791 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387198a9-d921-49d9-98be-eb657849c858 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:20.726670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387198a9-d921-49d9-98be-eb657849c858 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:20.731390 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff2e30-276c-4ed6-97fc-bad49cd2c169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:20.731990 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff2e30-276c-4ed6-97fc-bad49cd2c169 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:20.732270 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff2e30-276c-4ed6-97fc-bad49cd2c169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:20.732565 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff2e30-276c-4ed6-97fc-bad49cd2c169 admin admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:20.733239 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaff2e30-276c-4ed6-97fc-bad49cd2c169 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:21:20.733539 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaff2e30-276c-4ed6-97fc-bad49cd2c169 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:20.735189 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaff2e30-276c-4ed6-97fc-bad49cd2c169 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:20.750994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1324/2655] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:21:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:21:20.756044 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77bcda-9134-4080-b862-3cfd9536bf52 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:20.766768 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77bcda-9134-4080-b862-3cfd9536bf52 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:20.771731 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9424c-c0c9-4d32-95ba-44a7eec009ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:20.772593 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9424c-c0c9-4d32-95ba-44a7eec009ad admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:20.772983 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9424c-c0c9-4d32-95ba-44a7eec009ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:20.773273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9424c-c0c9-4d32-95ba-44a7eec009ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:20.774362 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f9424c-c0c9-4d32-95ba-44a7eec009ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:20.776254 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f9424c-c0c9-4d32-95ba-44a7eec009ad admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:20.842927 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1332/2656] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:21:20 2022] POST /identity/v3/projects => generated 385 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:21:20.846591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff2e30-276c-4ed6-97fc-bad49cd2c169 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:20.869703 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff2e30-276c-4ed6-97fc-bad49cd2c169 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:20.940339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c7ebb-23a9-403c-94a7-c31a447c4438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:20.940972 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c7ebb-23a9-403c-94a7-c31a447c4438 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:20.941313 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c7ebb-23a9-403c-94a7-c31a447c4438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:20.941590 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c7ebb-23a9-403c-94a7-c31a447c4438 admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:20.942357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871c7ebb-23a9-403c-94a7-c31a447c4438 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:20.944453 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871c7ebb-23a9-403c-94a7-c31a447c4438 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:20.967417 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1325/2657] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:21:20 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:21:20.972118 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9424c-c0c9-4d32-95ba-44a7eec009ad admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:20.984131 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9424c-c0c9-4d32-95ba-44a7eec009ad admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:20.988971 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e887bdd-a504-43d0-a8c9-c41f8e86fd37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:20.989673 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e887bdd-a504-43d0-a8c9-c41f8e86fd37 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:20.990057 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e887bdd-a504-43d0-a8c9-c41f8e86fd37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:20.990388 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e887bdd-a504-43d0-a8c9-c41f8e86fd37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:20.991155 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e887bdd-a504-43d0-a8c9-c41f8e86fd37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:20.992813 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e887bdd-a504-43d0-a8c9-c41f8e86fd37 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:21.004348 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1333/2658] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:21:21.008477 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c7ebb-23a9-403c-94a7-c31a447c4438 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:21.021208 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c7ebb-23a9-403c-94a7-c31a447c4438 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:21.026438 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5737b067-385e-4d26-a236-c8a84fcdcf32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:21.027191 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737b067-385e-4d26-a236-c8a84fcdcf32 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:21.027536 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737b067-385e-4d26-a236-c8a84fcdcf32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:21.027875 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737b067-385e-4d26-a236-c8a84fcdcf32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:21.028627 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5737b067-385e-4d26-a236-c8a84fcdcf32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:21.030373 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5737b067-385e-4d26-a236-c8a84fcdcf32 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:21.041817 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1326/2659] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:21:21.046648 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e887bdd-a504-43d0-a8c9-c41f8e86fd37 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:21.063429 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e887bdd-a504-43d0-a8c9-c41f8e86fd37 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:21.069054 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f32351-a088-4a14-9fc9-99aebec8f192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:21.069811 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f32351-a088-4a14-9fc9-99aebec8f192 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:21.070158 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f32351-a088-4a14-9fc9-99aebec8f192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:21.070500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f32351-a088-4a14-9fc9-99aebec8f192 admin admin] PATH_INFO: `/v3/projects/9afc5306d0454c8bbf47fab2309b076e/users/76c5be9087e543759060d309858ee12a/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:21.087407 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f32351-a088-4a14-9fc9-99aebec8f192 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9afc5306d0454c8bbf47fab2309b076e, user_id=76c5be9087e543759060d309858ee12a, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:21.090953 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f32351-a088-4a14-9fc9-99aebec8f192 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:21.105749 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1334/2660] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:21:21 2022] PUT /identity/v3/projects/9afc5306d0454c8bbf47fab2309b076e/users/76c5be9087e543759060d309858ee12a/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:21.117126 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43d54b-d518-42cc-a706-9592224bd488 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:21.117476 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43d54b-d518-42cc-a706-9592224bd488 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:21.117811 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43d54b-d518-42cc-a706-9592224bd488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:21.146240 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-ab43d54b-d518-42cc-a706-9592224bd488 None None] MFA Rules not processed for user `76c5be9087e543759060d309858ee12a`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:21.147560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-ab43d54b-d518-42cc-a706-9592224bd488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:21.175589 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:21.223427 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1327/2661] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:21.227851 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f32351-a088-4a14-9fc9-99aebec8f192 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:21.291539 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f32351-a088-4a14-9fc9-99aebec8f192 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:21.296791 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4578b-5d96-4a92-9d7e-5f8b47c17cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:21.297392 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4578b-5d96-4a92-9d7e-5f8b47c17cb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:21.297680 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4578b-5d96-4a92-9d7e-5f8b47c17cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:21.297950 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4578b-5d96-4a92-9d7e-5f8b47c17cb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:21.298594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc4578b-5d96-4a92-9d7e-5f8b47c17cb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:21.300527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc4578b-5d96-4a92-9d7e-5f8b47c17cb0 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:21.327704 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1335/2662] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:21:21 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:21:21.332169 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab43d54b-d518-42cc-a706-9592224bd488 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:21.344683 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab43d54b-d518-42cc-a706-9592224bd488 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:21.408186 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb26e9c-5138-4370-8099-72f250f47370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:21.408824 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb26e9c-5138-4370-8099-72f250f47370 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:21.409101 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb26e9c-5138-4370-8099-72f250f47370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:21.409372 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb26e9c-5138-4370-8099-72f250f47370 admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:21.410007 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb26e9c-5138-4370-8099-72f250f47370 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:21.411945 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb26e9c-5138-4370-8099-72f250f47370 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:21.439236 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1328/2663] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:21:21 2022] POST /identity/v3/users => generated 294 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:21:21.440375 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4578b-5d96-4a92-9d7e-5f8b47c17cb0 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:21.451197 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4578b-5d96-4a92-9d7e-5f8b47c17cb0 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:21.457043 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516afcd-d289-4c63-83da-7e3cfd682508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:21.457647 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516afcd-d289-4c63-83da-7e3cfd682508 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:21.457923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516afcd-d289-4c63-83da-7e3cfd682508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:21.458192 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516afcd-d289-4c63-83da-7e3cfd682508 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:21.458828 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e516afcd-d289-4c63-83da-7e3cfd682508 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:21.460560 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e516afcd-d289-4c63-83da-7e3cfd682508 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:21.474146 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1336/2664] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:21:21.478161 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb26e9c-5138-4370-8099-72f250f47370 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:21.489847 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb26e9c-5138-4370-8099-72f250f47370 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:21.505650 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81fe73b-1f3b-4b96-b9c3-c05655a81ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:21.508929 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fe73b-1f3b-4b96-b9c3-c05655a81ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:21.509314 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fe73b-1f3b-4b96-b9c3-c05655a81ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:21.509682 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fe73b-1f3b-4b96-b9c3-c05655a81ff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:21.510553 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81fe73b-1f3b-4b96-b9c3-c05655a81ff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:21.515480 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81fe73b-1f3b-4b96-b9c3-c05655a81ff8 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:21.530691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1329/2665] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:21:21.531986 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516afcd-d289-4c63-83da-7e3cfd682508 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:21.538442 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:21.542272 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516afcd-d289-4c63-83da-7e3cfd682508 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:21.549514 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46502a07-50e3-41e3-a950-a383314ca63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:21.550442 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46502a07-50e3-41e3-a950-a383314ca63c admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:21.551097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46502a07-50e3-41e3-a950-a383314ca63c admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:21.551611 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46502a07-50e3-41e3-a950-a383314ca63c admin admin] PATH_INFO: `/v3/projects/69a433e48d47482990f44ae5ed9f2b5e/users/c0dac950987c4af29b7422bb09857926/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:21.567335 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46502a07-50e3-41e3-a950-a383314ca63c admin admin] RBAC: Authorizing `identity:create_grant(project_id=69a433e48d47482990f44ae5ed9f2b5e, user_id=c0dac950987c4af29b7422bb09857926, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:21.570170 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46502a07-50e3-41e3-a950-a383314ca63c admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:21.582536 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1337/2666] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:21:21 2022] PUT /identity/v3/projects/69a433e48d47482990f44ae5ed9f2b5e/users/c0dac950987c4af29b7422bb09857926/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:21.594019 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf754a-0bf4-43a6-becd-7402986883a0 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:21.594537 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf754a-0bf4-43a6-becd-7402986883a0 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:21.594651 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf754a-0bf4-43a6-becd-7402986883a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:21.622548 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-06cf754a-0bf4-43a6-becd-7402986883a0 None None] MFA Rules not processed for user `c0dac950987c4af29b7422bb09857926`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:21.624024 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-06cf754a-0bf4-43a6-becd-7402986883a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:21.688500 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1330/2667] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:21.693095 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46502a07-50e3-41e3-a950-a383314ca63c admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:21.754762 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46502a07-50e3-41e3-a950-a383314ca63c admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:21.759430 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f435bd5-3e9c-4911-8c6f-722cbc7a6f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:21.760033 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f435bd5-3e9c-4911-8c6f-722cbc7a6f0e admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:21.760313 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f435bd5-3e9c-4911-8c6f-722cbc7a6f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:21.760583 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f435bd5-3e9c-4911-8c6f-722cbc7a6f0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:21.761248 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f435bd5-3e9c-4911-8c6f-722cbc7a6f0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:21.763667 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f435bd5-3e9c-4911-8c6f-722cbc7a6f0e admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:21.789652 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1338/2668] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:21:21 2022] POST /identity/v3/projects => generated 387 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:21:21.793835 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf754a-0bf4-43a6-becd-7402986883a0 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:21.804443 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf754a-0bf4-43a6-becd-7402986883a0 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:21.838326 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:21.870864 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406aa45-e8a7-4f28-aefe-b6916a6fe0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:21.873184 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406aa45-e8a7-4f28-aefe-b6916a6fe0b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:21.873184 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406aa45-e8a7-4f28-aefe-b6916a6fe0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:21.873184 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406aa45-e8a7-4f28-aefe-b6916a6fe0b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:21.873184 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1406aa45-e8a7-4f28-aefe-b6916a6fe0b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:21.875321 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1406aa45-e8a7-4f28-aefe-b6916a6fe0b8 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:21.900227 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1331/2669] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:21:21 2022] POST /identity/v3/users => generated 300 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:21:21.901479 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f435bd5-3e9c-4911-8c6f-722cbc7a6f0e admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:21.912350 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f435bd5-3e9c-4911-8c6f-722cbc7a6f0e admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:21.917062 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f43c99-c2cb-4649-8c77-3cc972bba5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:21.917653 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f43c99-c2cb-4649-8c77-3cc972bba5f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:21.917927 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f43c99-c2cb-4649-8c77-3cc972bba5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:21.918192 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f43c99-c2cb-4649-8c77-3cc972bba5f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:21.918886 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f43c99-c2cb-4649-8c77-3cc972bba5f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:21.920676 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f43c99-c2cb-4649-8c77-3cc972bba5f9 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:21.932434 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1339/2670] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:21:21.936682 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406aa45-e8a7-4f28-aefe-b6916a6fe0b8 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:21.946792 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406aa45-e8a7-4f28-aefe-b6916a6fe0b8 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:21.952194 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b95225-4fae-4032-b3d0-ce99d8be2922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:21.952880 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b95225-4fae-4032-b3d0-ce99d8be2922 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:21.953153 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b95225-4fae-4032-b3d0-ce99d8be2922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:21.953420 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b95225-4fae-4032-b3d0-ce99d8be2922 admin admin] PATH_INFO: `/v3/projects/b9a0cfabf99d4fa2bb1dea1e601d4ea0/users/c6952c40775b4d93a23de01ebdebdadf/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:21.969128 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b95225-4fae-4032-b3d0-ce99d8be2922 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9a0cfabf99d4fa2bb1dea1e601d4ea0, user_id=c6952c40775b4d93a23de01ebdebdadf, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:21.972092 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b95225-4fae-4032-b3d0-ce99d8be2922 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:21.985219 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1332/2671] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:21:21 2022] PUT /identity/v3/projects/b9a0cfabf99d4fa2bb1dea1e601d4ea0/users/c6952c40775b4d93a23de01ebdebdadf/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:21.989159 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f43c99-c2cb-4649-8c77-3cc972bba5f9 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:22.000593 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f43c99-c2cb-4649-8c77-3cc972bba5f9 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:22.054897 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa04c8-0f9f-42e1-8466-5a90de72f3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:22.055498 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa04c8-0f9f-42e1-8466-5a90de72f3da admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:22.055776 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa04c8-0f9f-42e1-8466-5a90de72f3da admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:22.056048 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa04c8-0f9f-42e1-8466-5a90de72f3da admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:22.056699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa04c8-0f9f-42e1-8466-5a90de72f3da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:22.058243 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa04c8-0f9f-42e1-8466-5a90de72f3da admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:22.069164 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1340/2672] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:21:22.072961 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b95225-4fae-4032-b3d0-ce99d8be2922 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:22.086379 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b95225-4fae-4032-b3d0-ce99d8be2922 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:22.094004 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677a23b-9735-49b4-9757-f66820fdd670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:22.094986 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677a23b-9735-49b4-9757-f66820fdd670 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:22.095486 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677a23b-9735-49b4-9757-f66820fdd670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:22.095973 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677a23b-9735-49b4-9757-f66820fdd670 admin admin] PATH_INFO: `/v3/projects/b9a0cfabf99d4fa2bb1dea1e601d4ea0/users/c6952c40775b4d93a23de01ebdebdadf/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:22.100778 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9677a23b-9735-49b4-9757-f66820fdd670 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9a0cfabf99d4fa2bb1dea1e601d4ea0, user_id=c6952c40775b4d93a23de01ebdebdadf, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:22.103993 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9677a23b-9735-49b4-9757-f66820fdd670 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:22.118304 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1333/2673] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:21:22 2022] PUT /identity/v3/projects/b9a0cfabf99d4fa2bb1dea1e601d4ea0/users/c6952c40775b4d93a23de01ebdebdadf/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:22.130314 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2148991-0c63-427d-946c-f942dd52d15b None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:22.130695 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2148991-0c63-427d-946c-f942dd52d15b None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:22.131074 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2148991-0c63-427d-946c-f942dd52d15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:22.162605 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-c2148991-0c63-427d-946c-f942dd52d15b None None] MFA Rules not processed for user `c6952c40775b4d93a23de01ebdebdadf`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:22.163838 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c2148991-0c63-427d-946c-f942dd52d15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:22.204010 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:22.233118 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1341/2674] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:22 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:22.244449 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677a23b-9735-49b4-9757-f66820fdd670 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:22.246148 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-9677a23b-9735-49b4-9757-f66820fdd670 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:22.306394 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677a23b-9735-49b4-9757-f66820fdd670 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:22.310709 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b629f-a62c-471d-b859-8ee202a73963 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:22.311530 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b629f-a62c-471d-b859-8ee202a73963 service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:22.311808 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b629f-a62c-471d-b859-8ee202a73963 service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:22.312073 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b629f-a62c-471d-b859-8ee202a73963 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:22.321170 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b629f-a62c-471d-b859-8ee202a73963 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:22.322606 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b629f-a62c-471d-b859-8ee202a73963 service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:22.332465 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1334/2675] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:21:22 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:21:22.987319 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2148991-0c63-427d-946c-f942dd52d15b None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:22.988888 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c2148991-0c63-427d-946c-f942dd52d15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:23.001863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2148991-0c63-427d-946c-f942dd52d15b None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:23.005714 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db469d-7c0b-43be-a72f-26271d750795 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:23.006305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db469d-7c0b-43be-a72f-26271d750795 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:23.006577 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db469d-7c0b-43be-a72f-26271d750795 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:23.006864 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db469d-7c0b-43be-a72f-26271d750795 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:23.070150 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db469d-7c0b-43be-a72f-26271d750795 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:23.072424 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db469d-7c0b-43be-a72f-26271d750795 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:23.083552 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1342/2676] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:21:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:21:25.176623 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b629f-a62c-471d-b859-8ee202a73963 service cinder] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:25.178802 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-d72b629f-a62c-471d-b859-8ee202a73963 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:25.276179 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b629f-a62c-471d-b859-8ee202a73963 service cinder] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:25.281406 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33444eb8-ab35-48b8-972b-718772d4dc2a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce508bb9aa164a7fb1831a9b5ff5b932', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:25.282008 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33444eb8-ab35-48b8-972b-718772d4dc2a service placement] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:25.282278 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33444eb8-ab35-48b8-972b-718772d4dc2a service placement] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:25.282542 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33444eb8-ab35-48b8-972b-718772d4dc2a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:25.284272 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-33444eb8-ab35-48b8-972b-718772d4dc2a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:25.297300 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33444eb8-ab35-48b8-972b-718772d4dc2a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:25.299300 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33444eb8-ab35-48b8-972b-718772d4dc2a service placement] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:25.305015 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:25.310936 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1335/2677] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:21:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:21:25.681844 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db469d-7c0b-43be-a72f-26271d750795 service cinder] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:25.683309 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-55db469d-7c0b-43be-a72f-26271d750795 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:25.756367 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db469d-7c0b-43be-a72f-26271d750795 service cinder] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:25.760957 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abdb8fa-6e25-4d4e-a3b2-7d357012946b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:25.761581 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdb8fa-6e25-4d4e-a3b2-7d357012946b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:25.761857 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdb8fa-6e25-4d4e-a3b2-7d357012946b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:25.762129 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdb8fa-6e25-4d4e-a3b2-7d357012946b admin admin] PATH_INFO: `/v3/users/3877b0d0d97743fb91fc3c2e8ba4006a` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:25.763283 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abdb8fa-6e25-4d4e-a3b2-7d357012946b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3877b0d0d97743fb91fc3c2e8ba4006a)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:25.765163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abdb8fa-6e25-4d4e-a3b2-7d357012946b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:25.822360 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-1abdb8fa-6e25-4d4e-a3b2-7d357012946b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3877b0d0d97743fb91fc3c2e8ba4006a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:25.831626 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-1abdb8fa-6e25-4d4e-a3b2-7d357012946b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3877b0d0d97743fb91fc3c2e8ba4006a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:25.832309 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-1abdb8fa-6e25-4d4e-a3b2-7d357012946b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3877b0d0d97743fb91fc3c2e8ba4006a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:25.832545 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-1abdb8fa-6e25-4d4e-a3b2-7d357012946b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3877b0d0d97743fb91fc3c2e8ba4006a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:25.840146 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-1abdb8fa-6e25-4d4e-a3b2-7d357012946b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3877b0d0d97743fb91fc3c2e8ba4006a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:25.852936 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:25.862668 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1343/2678] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:21:25 2022] DELETE /identity/v3/users/3877b0d0d97743fb91fc3c2e8ba4006a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:26.057540 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33444eb8-ab35-48b8-972b-718772d4dc2a service placement] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:26.068834 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33444eb8-ab35-48b8-972b-718772d4dc2a service placement] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:26.140097 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92800edb-c711-454f-b5c9-d70fc7e9a1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:26.140649 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92800edb-c711-454f-b5c9-d70fc7e9a1ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:26.140922 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92800edb-c711-454f-b5c9-d70fc7e9a1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:26.141187 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92800edb-c711-454f-b5c9-d70fc7e9a1ef admin admin] PATH_INFO: `/v3/projects/0412ec3ff65b4c7284b4aeba06d355dc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:26.142315 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92800edb-c711-454f-b5c9-d70fc7e9a1ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=0412ec3ff65b4c7284b4aeba06d355dc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:26.144224 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92800edb-c711-454f-b5c9-d70fc7e9a1ef admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:26.189603 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-92800edb-c711-454f-b5c9-d70fc7e9a1ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0412ec3ff65b4c7284b4aeba06d355dc', 'request_id': 'req-92800edb-c711-454f-b5c9-d70fc7e9a1ef'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:26.194979 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-92800edb-c711-454f-b5c9-d70fc7e9a1ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0412ec3ff65b4c7284b4aeba06d355dc', 'request_id': 'req-92800edb-c711-454f-b5c9-d70fc7e9a1ef'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:26.202758 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-92800edb-c711-454f-b5c9-d70fc7e9a1ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0412ec3ff65b4c7284b4aeba06d355dc', 'request_id': 'req-92800edb-c711-454f-b5c9-d70fc7e9a1ef'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:26.213860 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-92800edb-c711-454f-b5c9-d70fc7e9a1ef admin admin] The token cache is being invalidate because project 0412ec3ff65b4c7284b4aeba06d355dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:21:26.214147 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-92800edb-c711-454f-b5c9-d70fc7e9a1ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:26.214763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-92800edb-c711-454f-b5c9-d70fc7e9a1ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:26.216887 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1336/2679] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:21:26 2022] DELETE /identity/v3/projects/0412ec3ff65b4c7284b4aeba06d355dc => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:26.220842 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abdb8fa-6e25-4d4e-a3b2-7d357012946b admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:26.222210 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1abdb8fa-6e25-4d4e-a3b2-7d357012946b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:26.235988 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abdb8fa-6e25-4d4e-a3b2-7d357012946b admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:26.292304 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ace1dc-ab3f-490b-b170-9dda91499672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:26.292925 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ace1dc-ab3f-490b-b170-9dda91499672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:26.293202 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ace1dc-ab3f-490b-b170-9dda91499672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:26.293473 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ace1dc-ab3f-490b-b170-9dda91499672 admin admin] PATH_INFO: `/v3/users/140a107f2a4148c2acf5a56dc7d64355` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:26.294718 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ace1dc-ab3f-490b-b170-9dda91499672 admin admin] RBAC: Authorizing `identity:delete_user(user_id=140a107f2a4148c2acf5a56dc7d64355)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:26.296759 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ace1dc-ab3f-490b-b170-9dda91499672 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:26.342154 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c3ace1dc-ab3f-490b-b170-9dda91499672 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '140a107f2a4148c2acf5a56dc7d64355'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:26.349889 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c3ace1dc-ab3f-490b-b170-9dda91499672 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '140a107f2a4148c2acf5a56dc7d64355'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:26.350717 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c3ace1dc-ab3f-490b-b170-9dda91499672 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '140a107f2a4148c2acf5a56dc7d64355'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:26.351400 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c3ace1dc-ab3f-490b-b170-9dda91499672 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '140a107f2a4148c2acf5a56dc7d64355'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:26.360103 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c3ace1dc-ab3f-490b-b170-9dda91499672 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '140a107f2a4148c2acf5a56dc7d64355'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:26.381949 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1344/2680] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:21:26 2022] DELETE /identity/v3/users/140a107f2a4148c2acf5a56dc7d64355 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:26.559544 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92800edb-c711-454f-b5c9-d70fc7e9a1ef admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:26.561257 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-92800edb-c711-454f-b5c9-d70fc7e9a1ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:26.574356 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92800edb-c711-454f-b5c9-d70fc7e9a1ef admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:26.635042 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf3d85-94c1-4f31-b83c-7df1130c4778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:26.635672 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf3d85-94c1-4f31-b83c-7df1130c4778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:26.635969 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf3d85-94c1-4f31-b83c-7df1130c4778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:26.636264 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf3d85-94c1-4f31-b83c-7df1130c4778 admin admin] PATH_INFO: `/v3/projects/16fd2599e7514acf9864bf0bca9ac5f0` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:26.637968 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf3d85-94c1-4f31-b83c-7df1130c4778 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16fd2599e7514acf9864bf0bca9ac5f0)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:26.640255 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf3d85-94c1-4f31-b83c-7df1130c4778 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:26.680148 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:26.699323 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-6ecf3d85-94c1-4f31-b83c-7df1130c4778 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16fd2599e7514acf9864bf0bca9ac5f0', 'request_id': 'req-6ecf3d85-94c1-4f31-b83c-7df1130c4778'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:26.705219 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-6ecf3d85-94c1-4f31-b83c-7df1130c4778 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16fd2599e7514acf9864bf0bca9ac5f0', 'request_id': 'req-6ecf3d85-94c1-4f31-b83c-7df1130c4778'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:26.713618 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-6ecf3d85-94c1-4f31-b83c-7df1130c4778 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16fd2599e7514acf9864bf0bca9ac5f0', 'request_id': 'req-6ecf3d85-94c1-4f31-b83c-7df1130c4778'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:26.726123 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-6ecf3d85-94c1-4f31-b83c-7df1130c4778 admin admin] The token cache is being invalidate because project 16fd2599e7514acf9864bf0bca9ac5f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:21:26.726413 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-6ecf3d85-94c1-4f31-b83c-7df1130c4778 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:26.727251 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-6ecf3d85-94c1-4f31-b83c-7df1130c4778 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:26.728858 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1337/2681] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:21:26 2022] DELETE /identity/v3/projects/16fd2599e7514acf9864bf0bca9ac5f0 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:26.734721 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc82664-52dc-446d-9582-35f633c251ce None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:26.735267 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc82664-52dc-446d-9582-35f633c251ce None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:26.735515 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc82664-52dc-446d-9582-35f633c251ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:26.756235 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-cbc82664-52dc-446d-9582-35f633c251ce None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:26.757849 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-cbc82664-52dc-446d-9582-35f633c251ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:26.827186 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1345/2682] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:21:26.832149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7cdacc-a7e3-40d8-9c40-5ccd281a489b None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:26.832507 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7cdacc-a7e3-40d8-9c40-5ccd281a489b None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:26.832755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7cdacc-a7e3-40d8-9c40-5ccd281a489b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:26.854534 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-0d7cdacc-a7e3-40d8-9c40-5ccd281a489b None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:26.855844 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-0d7cdacc-a7e3-40d8-9c40-5ccd281a489b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:26.864614 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1338/2683] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:26.868984 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc82664-52dc-446d-9582-35f633c251ce None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:26.882923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc82664-52dc-446d-9582-35f633c251ce None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:26.887954 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de391e0-f64d-46b8-85f8-ccd771bc2933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:26.889534 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de391e0-f64d-46b8-85f8-ccd771bc2933 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:26.889534 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de391e0-f64d-46b8-85f8-ccd771bc2933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:26.889534 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de391e0-f64d-46b8-85f8-ccd771bc2933 admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:26.889815 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de391e0-f64d-46b8-85f8-ccd771bc2933 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:21:26.890018 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de391e0-f64d-46b8-85f8-ccd771bc2933 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:26.891771 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de391e0-f64d-46b8-85f8-ccd771bc2933 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:26.901940 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:26.909545 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1346/2684] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:21:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:21:26.912337 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7cdacc-a7e3-40d8-9c40-5ccd281a489b None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:26.922694 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7cdacc-a7e3-40d8-9c40-5ccd281a489b None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:26.928271 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0acd1-a3f1-40b6-a7b3-60805f36db82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:26.928957 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0acd1-a3f1-40b6-a7b3-60805f36db82 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:26.929297 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0acd1-a3f1-40b6-a7b3-60805f36db82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:26.929824 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0acd1-a3f1-40b6-a7b3-60805f36db82 admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:26.930547 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de0acd1-a3f1-40b6-a7b3-60805f36db82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:26.932594 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de0acd1-a3f1-40b6-a7b3-60805f36db82 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:26.968385 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1339/2685] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:21:26 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:21:26.970137 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de391e0-f64d-46b8-85f8-ccd771bc2933 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:26.987085 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de391e0-f64d-46b8-85f8-ccd771bc2933 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:27.052162 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765067f-0509-4bbd-a5f4-f412fbae3647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:27.052963 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765067f-0509-4bbd-a5f4-f412fbae3647 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:27.053393 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765067f-0509-4bbd-a5f4-f412fbae3647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:27.053813 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765067f-0509-4bbd-a5f4-f412fbae3647 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:27.054578 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c765067f-0509-4bbd-a5f4-f412fbae3647 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:27.056791 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c765067f-0509-4bbd-a5f4-f412fbae3647 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:27.106385 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1347/2686] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:21:26 2022] POST /identity/v3/users => generated 292 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:21:27.110995 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0acd1-a3f1-40b6-a7b3-60805f36db82 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:27.123490 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0acd1-a3f1-40b6-a7b3-60805f36db82 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:27.138145 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c09514-225d-43c2-95c1-923de2409e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:27.141843 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c09514-225d-43c2-95c1-923de2409e84 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:27.142123 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c09514-225d-43c2-95c1-923de2409e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:27.142392 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c09514-225d-43c2-95c1-923de2409e84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:27.144087 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c09514-225d-43c2-95c1-923de2409e84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:27.146814 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c09514-225d-43c2-95c1-923de2409e84 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:27.160143 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1340/2687] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:21:27.164379 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765067f-0509-4bbd-a5f4-f412fbae3647 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:27.177681 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765067f-0509-4bbd-a5f4-f412fbae3647 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:27.192604 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae836af-34cc-4053-9a82-f4068c1c2fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:27.193990 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae836af-34cc-4053-9a82-f4068c1c2fee admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:27.194487 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae836af-34cc-4053-9a82-f4068c1c2fee admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:27.195396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae836af-34cc-4053-9a82-f4068c1c2fee admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:27.196820 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae836af-34cc-4053-9a82-f4068c1c2fee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:27.200537 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae836af-34cc-4053-9a82-f4068c1c2fee admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:27.229625 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1348/2688] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:21:27.230950 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c09514-225d-43c2-95c1-923de2409e84 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:27.247191 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c09514-225d-43c2-95c1-923de2409e84 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:27.254604 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e053653-db5a-436e-bd9d-916c3d1bd701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:27.256552 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e053653-db5a-436e-bd9d-916c3d1bd701 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:27.257454 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e053653-db5a-436e-bd9d-916c3d1bd701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:27.258136 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e053653-db5a-436e-bd9d-916c3d1bd701 admin admin] PATH_INFO: `/v3/projects/0e7c6c045c04466e8a98ff8d02ecd9a7/users/40d013efa6ed411d858773433dd2394e/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:27.285571 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e053653-db5a-436e-bd9d-916c3d1bd701 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e7c6c045c04466e8a98ff8d02ecd9a7, user_id=40d013efa6ed411d858773433dd2394e, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:27.287375 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae836af-34cc-4053-9a82-f4068c1c2fee admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:27.289285 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-eae836af-34cc-4053-9a82-f4068c1c2fee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:27.290922 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e053653-db5a-436e-bd9d-916c3d1bd701 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:27.311089 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1341/2689] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:21:27 2022] PUT /identity/v3/projects/0e7c6c045c04466e8a98ff8d02ecd9a7/users/40d013efa6ed411d858773433dd2394e/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:27.326185 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051ddd7-dd88-4222-883a-dac191625d7e None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:27.326707 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051ddd7-dd88-4222-883a-dac191625d7e None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:27.327179 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051ddd7-dd88-4222-883a-dac191625d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:27.356945 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-c051ddd7-dd88-4222-883a-dac191625d7e None None] MFA Rules not processed for user `40d013efa6ed411d858773433dd2394e`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:27.358153 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-c051ddd7-dd88-4222-883a-dac191625d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:27.406133 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae836af-34cc-4053-9a82-f4068c1c2fee admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:27.441582 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1342/2690] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:21:27.469138 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0aa2ad-2939-4376-ba88-94abb22ea3a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:27.469751 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0aa2ad-2939-4376-ba88-94abb22ea3a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:27.470036 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0aa2ad-2939-4376-ba88-94abb22ea3a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:27.470305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0aa2ad-2939-4376-ba88-94abb22ea3a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:27.472125 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-6e0aa2ad-2939-4376-ba88-94abb22ea3a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:27.523153 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:27.539074 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0aa2ad-2939-4376-ba88-94abb22ea3a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:27.540504 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0aa2ad-2939-4376-ba88-94abb22ea3a0 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:27.553747 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1349/2691] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:21:27 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:21:27.566973 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c051ddd7-dd88-4222-883a-dac191625d7e None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:27.567654 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-c051ddd7-dd88-4222-883a-dac191625d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:27.584715 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c051ddd7-dd88-4222-883a-dac191625d7e None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:27.589964 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb2c87f-9d54-42f0-8079-9eb989293c43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:27.590701 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2c87f-9d54-42f0-8079-9eb989293c43 service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:27.591021 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2c87f-9d54-42f0-8079-9eb989293c43 service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:27.591296 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2c87f-9d54-42f0-8079-9eb989293c43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:27.601963 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb2c87f-9d54-42f0-8079-9eb989293c43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:27.603562 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb2c87f-9d54-42f0-8079-9eb989293c43 service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:27.613950 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1343/2692] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:21:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:21:31.388930 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c3154-b5e8-4b7f-b822-1f46b1fff14c None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:31.389400 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c3154-b5e8-4b7f-b822-1f46b1fff14c None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:31.389495 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c3154-b5e8-4b7f-b822-1f46b1fff14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:31.421549 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-149c3154-b5e8-4b7f-b822-1f46b1fff14c None None] MFA Rules not processed for user `1405e758a3af4e8ba659f498cd28ace1`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:31.422860 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-149c3154-b5e8-4b7f-b822-1f46b1fff14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:31.494556 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1350/2693] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Sep 12 14:21:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:21:31.544677 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb2c87f-9d54-42f0-8079-9eb989293c43 service cinder] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:31.548631 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-fcb2c87f-9d54-42f0-8079-9eb989293c43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:31.549637 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149c3154-b5e8-4b7f-b822-1f46b1fff14c None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:31.551245 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-149c3154-b5e8-4b7f-b822-1f46b1fff14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:31.586508 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:31.617366 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149c3154-b5e8-4b7f-b822-1f46b1fff14c None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:31.621922 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b5ffc7-67af-4144-b690-20db72478838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:31.622658 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5ffc7-67af-4144-b690-20db72478838 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:31.622972 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5ffc7-67af-4144-b690-20db72478838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:31.623278 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5ffc7-67af-4144-b690-20db72478838 admin admin] PATH_INFO: `/v3/users/76c5be9087e543759060d309858ee12a` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:31.623924 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb2c87f-9d54-42f0-8079-9eb989293c43 service cinder] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:31.624780 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b5ffc7-67af-4144-b690-20db72478838 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76c5be9087e543759060d309858ee12a)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:31.627115 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b5ffc7-67af-4144-b690-20db72478838 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:31.629937 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6992d9c-e196-43b4-a866-15f840c51dd0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a87ea9c3c5409b97a35c4857ae3ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:31.630597 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6992d9c-e196-43b4-a866-15f840c51dd0 service swift] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:31.630972 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6992d9c-e196-43b4-a866-15f840c51dd0 service swift] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:31.631148 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6992d9c-e196-43b4-a866-15f840c51dd0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:31.640579 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6992d9c-e196-43b4-a866-15f840c51dd0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:31.642176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6992d9c-e196-43b4-a866-15f840c51dd0 service swift] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:31.650356 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1344/2694] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Sep 12 14:21:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 14:21:31.683763 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-19b5ffc7-67af-4144-b690-20db72478838 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76c5be9087e543759060d309858ee12a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:31.692636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-19b5ffc7-67af-4144-b690-20db72478838 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76c5be9087e543759060d309858ee12a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:31.693369 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-19b5ffc7-67af-4144-b690-20db72478838 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76c5be9087e543759060d309858ee12a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:31.694154 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-19b5ffc7-67af-4144-b690-20db72478838 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76c5be9087e543759060d309858ee12a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:31.707624 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-19b5ffc7-67af-4144-b690-20db72478838 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76c5be9087e543759060d309858ee12a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:31.723339 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:31.731505 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1351/2695] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:21:31 2022] DELETE /identity/v3/users/76c5be9087e543759060d309858ee12a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:21:31.734195 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142134f1-f201-401b-b4bf-1bdaf6a02765 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:31.734819 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142134f1-f201-401b-b4bf-1bdaf6a02765 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:31.735387 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142134f1-f201-401b-b4bf-1bdaf6a02765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:31.763203 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-142134f1-f201-401b-b4bf-1bdaf6a02765 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:31.765003 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-142134f1-f201-401b-b4bf-1bdaf6a02765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:31.855945 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1345/2696] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:31.864377 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b5ffc7-67af-4144-b690-20db72478838 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:31.866144 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-19b5ffc7-67af-4144-b690-20db72478838 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:31.935315 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b5ffc7-67af-4144-b690-20db72478838 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:31.939239 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b5d23-f861-4dce-ad25-ebfa2025fc1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:31.939868 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b5d23-f861-4dce-ad25-ebfa2025fc1e service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:31.940147 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b5d23-f861-4dce-ad25-ebfa2025fc1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:31.940421 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b5d23-f861-4dce-ad25-ebfa2025fc1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:31.951024 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738b5d23-f861-4dce-ad25-ebfa2025fc1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:31.952493 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738b5d23-f861-4dce-ad25-ebfa2025fc1e service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:31.962519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1352/2697] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:21:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:21:32.284556 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142134f1-f201-401b-b4bf-1bdaf6a02765 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:32.287996 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-142134f1-f201-401b-b4bf-1bdaf6a02765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:32.302131 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142134f1-f201-401b-b4bf-1bdaf6a02765 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:32.341155 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a2e43-a1b9-4e23-98ba-d3d0fc060b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:32.342365 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a2e43-a1b9-4e23-98ba-d3d0fc060b45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:32.342957 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a2e43-a1b9-4e23-98ba-d3d0fc060b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:32.343529 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a2e43-a1b9-4e23-98ba-d3d0fc060b45 admin admin] PATH_INFO: `/v3/projects/9afc5306d0454c8bbf47fab2309b076e` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:32.344889 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a2e43-a1b9-4e23-98ba-d3d0fc060b45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9afc5306d0454c8bbf47fab2309b076e)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:32.347156 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a2e43-a1b9-4e23-98ba-d3d0fc060b45 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:32.400848 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-5b6a2e43-a1b9-4e23-98ba-d3d0fc060b45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9afc5306d0454c8bbf47fab2309b076e', 'request_id': 'req-5b6a2e43-a1b9-4e23-98ba-d3d0fc060b45'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:32.407064 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-5b6a2e43-a1b9-4e23-98ba-d3d0fc060b45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9afc5306d0454c8bbf47fab2309b076e', 'request_id': 'req-5b6a2e43-a1b9-4e23-98ba-d3d0fc060b45'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:32.417348 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-5b6a2e43-a1b9-4e23-98ba-d3d0fc060b45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9afc5306d0454c8bbf47fab2309b076e', 'request_id': 'req-5b6a2e43-a1b9-4e23-98ba-d3d0fc060b45'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:32.432148 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-5b6a2e43-a1b9-4e23-98ba-d3d0fc060b45 admin admin] The token cache is being invalidate because project 9afc5306d0454c8bbf47fab2309b076e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:21:32.432559 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-5b6a2e43-a1b9-4e23-98ba-d3d0fc060b45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:32.433216 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-5b6a2e43-a1b9-4e23-98ba-d3d0fc060b45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:32.436362 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1346/2698] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:21:32 2022] DELETE /identity/v3/projects/9afc5306d0454c8bbf47fab2309b076e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:32.440175 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b5d23-f861-4dce-ad25-ebfa2025fc1e service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:32.451246 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b5d23-f861-4dce-ad25-ebfa2025fc1e service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:32.517641 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a83ae-2fec-40df-90f7-477f1df28500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:32.518560 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a83ae-2fec-40df-90f7-477f1df28500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:32.518944 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a83ae-2fec-40df-90f7-477f1df28500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:32.519251 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a83ae-2fec-40df-90f7-477f1df28500 admin admin] PATH_INFO: `/v3/users/c0dac950987c4af29b7422bb09857926` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:32.520443 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045a83ae-2fec-40df-90f7-477f1df28500 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0dac950987c4af29b7422bb09857926)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:32.522702 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045a83ae-2fec-40df-90f7-477f1df28500 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:32.580059 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-045a83ae-2fec-40df-90f7-477f1df28500 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0dac950987c4af29b7422bb09857926'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:32.592251 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-045a83ae-2fec-40df-90f7-477f1df28500 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0dac950987c4af29b7422bb09857926'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:32.593186 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-045a83ae-2fec-40df-90f7-477f1df28500 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0dac950987c4af29b7422bb09857926'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:32.594019 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-045a83ae-2fec-40df-90f7-477f1df28500 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0dac950987c4af29b7422bb09857926'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:32.603571 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-045a83ae-2fec-40df-90f7-477f1df28500 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0dac950987c4af29b7422bb09857926'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:32.628969 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1353/2699] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:21:32 2022] DELETE /identity/v3/users/c0dac950987c4af29b7422bb09857926 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:32.935731 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a2e43-a1b9-4e23-98ba-d3d0fc060b45 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:32.937398 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-5b6a2e43-a1b9-4e23-98ba-d3d0fc060b45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:32.954707 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a2e43-a1b9-4e23-98ba-d3d0fc060b45 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:33.007893 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:33.027133 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aa9dad-4a88-4063-b843-c90a60e98656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:33.027948 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa9dad-4a88-4063-b843-c90a60e98656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:33.028342 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa9dad-4a88-4063-b843-c90a60e98656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:33.028713 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa9dad-4a88-4063-b843-c90a60e98656 admin admin] PATH_INFO: `/v3/projects/69a433e48d47482990f44ae5ed9f2b5e` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:33.029932 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aa9dad-4a88-4063-b843-c90a60e98656 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69a433e48d47482990f44ae5ed9f2b5e)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:33.032118 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aa9dad-4a88-4063-b843-c90a60e98656 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:33.084252 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-01aa9dad-4a88-4063-b843-c90a60e98656 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69a433e48d47482990f44ae5ed9f2b5e', 'request_id': 'req-01aa9dad-4a88-4063-b843-c90a60e98656'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:33.091628 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-01aa9dad-4a88-4063-b843-c90a60e98656 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69a433e48d47482990f44ae5ed9f2b5e', 'request_id': 'req-01aa9dad-4a88-4063-b843-c90a60e98656'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:33.101233 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-01aa9dad-4a88-4063-b843-c90a60e98656 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69a433e48d47482990f44ae5ed9f2b5e', 'request_id': 'req-01aa9dad-4a88-4063-b843-c90a60e98656'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:33.118092 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-01aa9dad-4a88-4063-b843-c90a60e98656 admin admin] The token cache is being invalidate because project 69a433e48d47482990f44ae5ed9f2b5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:21:33.118463 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-01aa9dad-4a88-4063-b843-c90a60e98656 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:33.119189 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-01aa9dad-4a88-4063-b843-c90a60e98656 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:33.120760 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1347/2700] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:21:32 2022] DELETE /identity/v3/projects/69a433e48d47482990f44ae5ed9f2b5e => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:33.125475 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a83ae-2fec-40df-90f7-477f1df28500 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:33.127061 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-045a83ae-2fec-40df-90f7-477f1df28500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:33.140344 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a83ae-2fec-40df-90f7-477f1df28500 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:33.159328 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:33.205959 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8dacd5-9af2-4f86-8fb0-7366e101cefa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:33.206520 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8dacd5-9af2-4f86-8fb0-7366e101cefa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:33.206914 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8dacd5-9af2-4f86-8fb0-7366e101cefa admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:33.207141 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8dacd5-9af2-4f86-8fb0-7366e101cefa admin admin] PATH_INFO: `/v3/users/c6952c40775b4d93a23de01ebdebdadf` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:33.208279 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8dacd5-9af2-4f86-8fb0-7366e101cefa admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6952c40775b4d93a23de01ebdebdadf)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:33.210185 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8dacd5-9af2-4f86-8fb0-7366e101cefa admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:33.263918 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-bb8dacd5-9af2-4f86-8fb0-7366e101cefa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6952c40775b4d93a23de01ebdebdadf'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:33.273543 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-bb8dacd5-9af2-4f86-8fb0-7366e101cefa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6952c40775b4d93a23de01ebdebdadf'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:33.274376 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-bb8dacd5-9af2-4f86-8fb0-7366e101cefa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6952c40775b4d93a23de01ebdebdadf'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:33.275039 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-bb8dacd5-9af2-4f86-8fb0-7366e101cefa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6952c40775b4d93a23de01ebdebdadf'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:33.283951 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-bb8dacd5-9af2-4f86-8fb0-7366e101cefa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6952c40775b4d93a23de01ebdebdadf'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:33.306149 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1354/2701] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:21:33 2022] DELETE /identity/v3/users/c6952c40775b4d93a23de01ebdebdadf => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:21:33.632124 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aa9dad-4a88-4063-b843-c90a60e98656 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:33.637640 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-01aa9dad-4a88-4063-b843-c90a60e98656 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:33.670465 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aa9dad-4a88-4063-b843-c90a60e98656 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:33.756425 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d7ce20-c44d-47ca-8419-fbcc080e2a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:33.757337 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7ce20-c44d-47ca-8419-fbcc080e2a21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:33.757720 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7ce20-c44d-47ca-8419-fbcc080e2a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:33.758089 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7ce20-c44d-47ca-8419-fbcc080e2a21 admin admin] PATH_INFO: `/v3/projects/b9a0cfabf99d4fa2bb1dea1e601d4ea0` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:33.759400 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d7ce20-c44d-47ca-8419-fbcc080e2a21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9a0cfabf99d4fa2bb1dea1e601d4ea0)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:33.761659 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d7ce20-c44d-47ca-8419-fbcc080e2a21 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:33.822220 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-90d7ce20-c44d-47ca-8419-fbcc080e2a21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9a0cfabf99d4fa2bb1dea1e601d4ea0', 'request_id': 'req-90d7ce20-c44d-47ca-8419-fbcc080e2a21'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:33.828414 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-90d7ce20-c44d-47ca-8419-fbcc080e2a21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9a0cfabf99d4fa2bb1dea1e601d4ea0', 'request_id': 'req-90d7ce20-c44d-47ca-8419-fbcc080e2a21'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:33.843174 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-90d7ce20-c44d-47ca-8419-fbcc080e2a21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9a0cfabf99d4fa2bb1dea1e601d4ea0', 'request_id': 'req-90d7ce20-c44d-47ca-8419-fbcc080e2a21'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:33.856667 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-90d7ce20-c44d-47ca-8419-fbcc080e2a21 admin admin] The token cache is being invalidate because project b9a0cfabf99d4fa2bb1dea1e601d4ea0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:21:33.857056 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-90d7ce20-c44d-47ca-8419-fbcc080e2a21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:33.857836 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-90d7ce20-c44d-47ca-8419-fbcc080e2a21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:33.863747 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1348/2702] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:21:33 2022] DELETE /identity/v3/projects/b9a0cfabf99d4fa2bb1dea1e601d4ea0 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:33.866178 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dafc3c9-33dd-4f74-839f-102632e3baf7 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:33.867159 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dafc3c9-33dd-4f74-839f-102632e3baf7 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:33.867804 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dafc3c9-33dd-4f74-839f-102632e3baf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:33.892448 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-7dafc3c9-33dd-4f74-839f-102632e3baf7 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:33.894552 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7dafc3c9-33dd-4f74-839f-102632e3baf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:33.978277 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1355/2703] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:33.987048 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf5b278-c300-4f74-840c-ac017f1da1b2 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:33.988048 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf5b278-c300-4f74-840c-ac017f1da1b2 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:33.988048 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf5b278-c300-4f74-840c-ac017f1da1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:34.004322 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:34.012689 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-5bf5b278-c300-4f74-840c-ac017f1da1b2 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:34.015182 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-5bf5b278-c300-4f74-840c-ac017f1da1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:34.027559 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1349/2704] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:34.032997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dafc3c9-33dd-4f74-839f-102632e3baf7 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:34.043105 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dafc3c9-33dd-4f74-839f-102632e3baf7 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:34.047865 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f89e8a-c34d-480c-b903-9a7bb67d675f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:34.048474 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f89e8a-c34d-480c-b903-9a7bb67d675f admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:34.048760 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f89e8a-c34d-480c-b903-9a7bb67d675f admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:34.049032 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f89e8a-c34d-480c-b903-9a7bb67d675f admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:34.049714 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f89e8a-c34d-480c-b903-9a7bb67d675f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:21:34.050138 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f89e8a-c34d-480c-b903-9a7bb67d675f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:34.051760 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f89e8a-c34d-480c-b903-9a7bb67d675f admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:34.065100 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1356/2705] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:21:34.070237 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf5b278-c300-4f74-840c-ac017f1da1b2 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:34.081803 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf5b278-c300-4f74-840c-ac017f1da1b2 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:34.088512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9942e-e1ea-4dfc-9e19-22f099fe6c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:34.089288 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9942e-e1ea-4dfc-9e19-22f099fe6c85 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:34.089938 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9942e-e1ea-4dfc-9e19-22f099fe6c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:34.090516 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9942e-e1ea-4dfc-9e19-22f099fe6c85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:34.091288 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b9942e-e1ea-4dfc-9e19-22f099fe6c85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:34.093856 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b9942e-e1ea-4dfc-9e19-22f099fe6c85 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:34.129545 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1350/2706] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:21:34 2022] POST /identity/v3/projects => generated 391 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:21:34.134219 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f89e8a-c34d-480c-b903-9a7bb67d675f admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:34.146142 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f89e8a-c34d-480c-b903-9a7bb67d675f admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:34.175023 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:34.220356 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ce591-d37a-4ee2-9d4e-4dcaef8216bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:34.221109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ce591-d37a-4ee2-9d4e-4dcaef8216bd admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:34.221399 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ce591-d37a-4ee2-9d4e-4dcaef8216bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:34.221673 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ce591-d37a-4ee2-9d4e-4dcaef8216bd admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:34.222348 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ce591-d37a-4ee2-9d4e-4dcaef8216bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:34.224306 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ce591-d37a-4ee2-9d4e-4dcaef8216bd admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:34.252170 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1357/2707] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:21:34 2022] POST /identity/v3/users => generated 296 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:21:34.256043 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9942e-e1ea-4dfc-9e19-22f099fe6c85 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:34.267541 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9942e-e1ea-4dfc-9e19-22f099fe6c85 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:34.273293 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b49ee-c60b-4520-8390-cc84a91b7de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:34.274017 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b49ee-c60b-4520-8390-cc84a91b7de5 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:34.274393 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b49ee-c60b-4520-8390-cc84a91b7de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:34.274781 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b49ee-c60b-4520-8390-cc84a91b7de5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:34.275700 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b49ee-c60b-4520-8390-cc84a91b7de5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:34.277302 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b49ee-c60b-4520-8390-cc84a91b7de5 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:34.290874 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1351/2708] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:21:34.292374 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ce591-d37a-4ee2-9d4e-4dcaef8216bd admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:34.303589 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ce591-d37a-4ee2-9d4e-4dcaef8216bd admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:34.310696 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f574da2-4083-4901-89d2-164bc1773d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:34.311828 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f574da2-4083-4901-89d2-164bc1773d50 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:34.312110 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f574da2-4083-4901-89d2-164bc1773d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:34.312385 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f574da2-4083-4901-89d2-164bc1773d50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:34.313049 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f574da2-4083-4901-89d2-164bc1773d50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:34.315038 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f574da2-4083-4901-89d2-164bc1773d50 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:34.329675 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1358/2709] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:21:34.336428 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b49ee-c60b-4520-8390-cc84a91b7de5 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:34.348862 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b49ee-c60b-4520-8390-cc84a91b7de5 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:34.353881 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163482f-7552-4220-8f2e-cb42661a8a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:34.354575 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163482f-7552-4220-8f2e-cb42661a8a3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:34.354972 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163482f-7552-4220-8f2e-cb42661a8a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:34.355173 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163482f-7552-4220-8f2e-cb42661a8a3b admin admin] PATH_INFO: `/v3/projects/aa0ed12cfa6a44bea6d6894143b8ba71/users/44bb8b34af2240cea047bfbcdcf9e9ab/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:34.373449 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163482f-7552-4220-8f2e-cb42661a8a3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa0ed12cfa6a44bea6d6894143b8ba71, user_id=44bb8b34af2240cea047bfbcdcf9e9ab, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:34.376240 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163482f-7552-4220-8f2e-cb42661a8a3b admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:34.407096 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1352/2710] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:21:34 2022] PUT /identity/v3/projects/aa0ed12cfa6a44bea6d6894143b8ba71/users/44bb8b34af2240cea047bfbcdcf9e9ab/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:34.420967 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde6f4f-5b1c-44ae-b0ee-1aa8927ad3d4 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:34.421368 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde6f4f-5b1c-44ae-b0ee-1aa8927ad3d4 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:34.421692 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde6f4f-5b1c-44ae-b0ee-1aa8927ad3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:34.454304 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-9fde6f4f-5b1c-44ae-b0ee-1aa8927ad3d4 None None] MFA Rules not processed for user `44bb8b34af2240cea047bfbcdcf9e9ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:34.455595 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-9fde6f4f-5b1c-44ae-b0ee-1aa8927ad3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:34.528511 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1359/2711] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:34 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:34.535533 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163482f-7552-4220-8f2e-cb42661a8a3b admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:34.603708 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163482f-7552-4220-8f2e-cb42661a8a3b admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:34.611432 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae89e8-8f41-4148-b1a7-65c3957d2f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:34.612161 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae89e8-8f41-4148-b1a7-65c3957d2f0f admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:34.612506 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae89e8-8f41-4148-b1a7-65c3957d2f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:34.612846 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae89e8-8f41-4148-b1a7-65c3957d2f0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:34.613574 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ae89e8-8f41-4148-b1a7-65c3957d2f0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:34.615613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ae89e8-8f41-4148-b1a7-65c3957d2f0f admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:34.650709 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1353/2712] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:21:34 2022] POST /identity/v3/projects => generated 393 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:21:34.655067 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde6f4f-5b1c-44ae-b0ee-1aa8927ad3d4 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:34.667647 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde6f4f-5b1c-44ae-b0ee-1aa8927ad3d4 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:34.742660 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d384d-1d0f-4c47-9af5-37c6354f7b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:34.743333 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d384d-1d0f-4c47-9af5-37c6354f7b53 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:34.743603 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d384d-1d0f-4c47-9af5-37c6354f7b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:34.743878 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d384d-1d0f-4c47-9af5-37c6354f7b53 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:34.744520 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d384d-1d0f-4c47-9af5-37c6354f7b53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:34.748498 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d384d-1d0f-4c47-9af5-37c6354f7b53 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:34.775419 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1360/2713] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:21:34 2022] POST /identity/v3/users => generated 303 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:21:34.779833 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae89e8-8f41-4148-b1a7-65c3957d2f0f admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:34.790741 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae89e8-8f41-4148-b1a7-65c3957d2f0f admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:34.796428 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222d4fe-5b54-45ba-a5d2-9541d832cd9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:34.797142 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222d4fe-5b54-45ba-a5d2-9541d832cd9d admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:34.797515 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222d4fe-5b54-45ba-a5d2-9541d832cd9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:34.797852 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222d4fe-5b54-45ba-a5d2-9541d832cd9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:34.798579 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0222d4fe-5b54-45ba-a5d2-9541d832cd9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:34.800433 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0222d4fe-5b54-45ba-a5d2-9541d832cd9d admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:34.818585 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1354/2714] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:21:34.819842 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d384d-1d0f-4c47-9af5-37c6354f7b53 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:34.836946 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d384d-1d0f-4c47-9af5-37c6354f7b53 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:34.842299 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599581e-7157-46af-ac1f-5ba979f8a8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:34.843000 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599581e-7157-46af-ac1f-5ba979f8a8d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:34.843283 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599581e-7157-46af-ac1f-5ba979f8a8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:34.843559 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599581e-7157-46af-ac1f-5ba979f8a8d6 admin admin] PATH_INFO: `/v3/projects/4d2c0afcd92a49529027cff8f253f40b/users/3f2f573644ff40aaab55127b18cb83a6/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:34.861621 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3599581e-7157-46af-ac1f-5ba979f8a8d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d2c0afcd92a49529027cff8f253f40b, user_id=3f2f573644ff40aaab55127b18cb83a6, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:34.864345 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3599581e-7157-46af-ac1f-5ba979f8a8d6 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:34.876915 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:34.882518 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1361/2715] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:21:34 2022] PUT /identity/v3/projects/4d2c0afcd92a49529027cff8f253f40b/users/3f2f573644ff40aaab55127b18cb83a6/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:34.886633 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222d4fe-5b54-45ba-a5d2-9541d832cd9d admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:34.899556 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222d4fe-5b54-45ba-a5d2-9541d832cd9d admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:34.969669 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4cb4e6-f262-40b7-b68e-45c26a787d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:34.970590 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4cb4e6-f262-40b7-b68e-45c26a787d4e admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:34.970899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4cb4e6-f262-40b7-b68e-45c26a787d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:34.971199 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4cb4e6-f262-40b7-b68e-45c26a787d4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:34.971790 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4cb4e6-f262-40b7-b68e-45c26a787d4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:34.973358 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4cb4e6-f262-40b7-b68e-45c26a787d4e admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:34.989822 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1355/2716] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:34 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:21:34.991322 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599581e-7157-46af-ac1f-5ba979f8a8d6 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:35.004102 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599581e-7157-46af-ac1f-5ba979f8a8d6 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:35.010327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d763d-bd39-4cdf-9fff-6bf819498c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:35.011333 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d763d-bd39-4cdf-9fff-6bf819498c2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:35.011693 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d763d-bd39-4cdf-9fff-6bf819498c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:35.012079 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d763d-bd39-4cdf-9fff-6bf819498c2b admin admin] PATH_INFO: `/v3/projects/4d2c0afcd92a49529027cff8f253f40b/users/3f2f573644ff40aaab55127b18cb83a6/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:35.014116 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10d763d-bd39-4cdf-9fff-6bf819498c2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d2c0afcd92a49529027cff8f253f40b, user_id=3f2f573644ff40aaab55127b18cb83a6, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:35.017729 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10d763d-bd39-4cdf-9fff-6bf819498c2b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:35.030699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1362/2717] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:21:34 2022] PUT /identity/v3/projects/4d2c0afcd92a49529027cff8f253f40b/users/3f2f573644ff40aaab55127b18cb83a6/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:35.043915 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da81f9a-6863-4f4a-b80c-f55dc8a15148 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:35.044181 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da81f9a-6863-4f4a-b80c-f55dc8a15148 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:35.044583 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da81f9a-6863-4f4a-b80c-f55dc8a15148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:35.074009 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:35.077466 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-0da81f9a-6863-4f4a-b80c-f55dc8a15148 None None] MFA Rules not processed for user `3f2f573644ff40aaab55127b18cb83a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:35.079868 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-0da81f9a-6863-4f4a-b80c-f55dc8a15148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:35.285186 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1356/2718] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:35 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:35.296681 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d763d-bd39-4cdf-9fff-6bf819498c2b admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:35.298416 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d10d763d-bd39-4cdf-9fff-6bf819498c2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:35.373210 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d763d-bd39-4cdf-9fff-6bf819498c2b admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:35.377103 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd26086-612d-4c84-b6fe-b704b2ebad17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:35.377866 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd26086-612d-4c84-b6fe-b704b2ebad17 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:35.378233 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd26086-612d-4c84-b6fe-b704b2ebad17 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:35.378614 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd26086-612d-4c84-b6fe-b704b2ebad17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:35.390349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd26086-612d-4c84-b6fe-b704b2ebad17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:35.392064 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd26086-612d-4c84-b6fe-b704b2ebad17 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:35.403762 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1363/2719] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:21:35 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:21:35.462884 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da81f9a-6863-4f4a-b80c-f55dc8a15148 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:35.474334 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da81f9a-6863-4f4a-b80c-f55dc8a15148 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:35.479650 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35464029-784a-4a2f-af1f-6c9727519d86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:35.480261 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35464029-784a-4a2f-af1f-6c9727519d86 service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:35.480412 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35464029-784a-4a2f-af1f-6c9727519d86 service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:35.481128 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35464029-784a-4a2f-af1f-6c9727519d86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:35.548626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35464029-784a-4a2f-af1f-6c9727519d86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:35.550309 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35464029-784a-4a2f-af1f-6c9727519d86 service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:35.562775 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1357/2720] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:21:35 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:21:38.513363 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd26086-612d-4c84-b6fe-b704b2ebad17 service cinder] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:38.516917 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-efd26086-612d-4c84-b6fe-b704b2ebad17 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:38.596808 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd26086-612d-4c84-b6fe-b704b2ebad17 service cinder] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:38.607822 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3dc82-ffd9-412d-811d-3ad11529b519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:38.608648 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3dc82-ffd9-412d-811d-3ad11529b519 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:38.608921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3dc82-ffd9-412d-811d-3ad11529b519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:38.609187 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3dc82-ffd9-412d-811d-3ad11529b519 admin admin] PATH_INFO: `/v3/users/40d013efa6ed411d858773433dd2394e` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:38.610355 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb3dc82-ffd9-412d-811d-3ad11529b519 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40d013efa6ed411d858773433dd2394e)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:38.612308 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb3dc82-ffd9-412d-811d-3ad11529b519 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:38.670454 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7eb3dc82-ffd9-412d-811d-3ad11529b519 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40d013efa6ed411d858773433dd2394e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:38.684423 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7eb3dc82-ffd9-412d-811d-3ad11529b519 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40d013efa6ed411d858773433dd2394e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:38.685696 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7eb3dc82-ffd9-412d-811d-3ad11529b519 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40d013efa6ed411d858773433dd2394e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:38.686550 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7eb3dc82-ffd9-412d-811d-3ad11529b519 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40d013efa6ed411d858773433dd2394e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:38.697431 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7eb3dc82-ffd9-412d-811d-3ad11529b519 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40d013efa6ed411d858773433dd2394e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:38.725511 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1364/2721] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:21:38 2022] DELETE /identity/v3/users/40d013efa6ed411d858773433dd2394e => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:38.729879 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac71b69f-5fc9-4f9b-a4d7-18a49c2a55a2 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:38.730425 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac71b69f-5fc9-4f9b-a4d7-18a49c2a55a2 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:38.731063 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac71b69f-5fc9-4f9b-a4d7-18a49c2a55a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:38.752417 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-ac71b69f-5fc9-4f9b-a4d7-18a49c2a55a2 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:38.753626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-ac71b69f-5fc9-4f9b-a4d7-18a49c2a55a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:38.827375 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1358/2722] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:38.839396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3dc82-ffd9-412d-811d-3ad11529b519 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:38.841624 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7eb3dc82-ffd9-412d-811d-3ad11529b519 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:38.856009 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:38.945749 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3dc82-ffd9-412d-811d-3ad11529b519 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:38.950171 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97df37e2-fa67-4189-8454-7a348e5a95e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:38.950784 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df37e2-fa67-4189-8454-7a348e5a95e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:38.951105 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df37e2-fa67-4189-8454-7a348e5a95e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:38.951375 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df37e2-fa67-4189-8454-7a348e5a95e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:38.963652 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97df37e2-fa67-4189-8454-7a348e5a95e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:38.965682 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97df37e2-fa67-4189-8454-7a348e5a95e0 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:38.975706 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1365/2723] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:21:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:21:39.278991 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac71b69f-5fc9-4f9b-a4d7-18a49c2a55a2 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:39.280877 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-ac71b69f-5fc9-4f9b-a4d7-18a49c2a55a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:39.297751 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac71b69f-5fc9-4f9b-a4d7-18a49c2a55a2 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:39.299220 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97df37e2-fa67-4189-8454-7a348e5a95e0 service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:39.300920 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-97df37e2-fa67-4189-8454-7a348e5a95e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:39.305302 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd7272-3c5a-4d0a-b028-5c66948a7f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:39.306273 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd7272-3c5a-4d0a-b028-5c66948a7f0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:39.306551 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd7272-3c5a-4d0a-b028-5c66948a7f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:39.307015 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd7272-3c5a-4d0a-b028-5c66948a7f0a admin admin] PATH_INFO: `/v3/users/44bb8b34af2240cea047bfbcdcf9e9ab` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:39.309096 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd7272-3c5a-4d0a-b028-5c66948a7f0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=44bb8b34af2240cea047bfbcdcf9e9ab)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:39.312075 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd7272-3c5a-4d0a-b028-5c66948a7f0a admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:39.323271 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97df37e2-fa67-4189-8454-7a348e5a95e0 service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:39.330470 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2673ded-91dc-4947-9d06-92a4983afee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:39.332947 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2673ded-91dc-4947-9d06-92a4983afee4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:39.333862 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2673ded-91dc-4947-9d06-92a4983afee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:39.334787 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2673ded-91dc-4947-9d06-92a4983afee4 admin admin] PATH_INFO: `/v3/projects/0e7c6c045c04466e8a98ff8d02ecd9a7` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:39.337585 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2673ded-91dc-4947-9d06-92a4983afee4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e7c6c045c04466e8a98ff8d02ecd9a7)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:39.340819 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2673ded-91dc-4947-9d06-92a4983afee4 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:39.402219 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:39.408457 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a2673ded-91dc-4947-9d06-92a4983afee4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e7c6c045c04466e8a98ff8d02ecd9a7', 'request_id': 'req-a2673ded-91dc-4947-9d06-92a4983afee4'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:39.419231 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a2673ded-91dc-4947-9d06-92a4983afee4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e7c6c045c04466e8a98ff8d02ecd9a7', 'request_id': 'req-a2673ded-91dc-4947-9d06-92a4983afee4'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:39.428617 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-30cd7272-3c5a-4d0a-b028-5c66948a7f0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44bb8b34af2240cea047bfbcdcf9e9ab'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:39.431499 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a2673ded-91dc-4947-9d06-92a4983afee4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e7c6c045c04466e8a98ff8d02ecd9a7', 'request_id': 'req-a2673ded-91dc-4947-9d06-92a4983afee4'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:39.440084 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-30cd7272-3c5a-4d0a-b028-5c66948a7f0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44bb8b34af2240cea047bfbcdcf9e9ab'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:39.440806 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-30cd7272-3c5a-4d0a-b028-5c66948a7f0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44bb8b34af2240cea047bfbcdcf9e9ab'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:39.441471 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-30cd7272-3c5a-4d0a-b028-5c66948a7f0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44bb8b34af2240cea047bfbcdcf9e9ab'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:39.447674 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a2673ded-91dc-4947-9d06-92a4983afee4 admin admin] The token cache is being invalidate because project 0e7c6c045c04466e8a98ff8d02ecd9a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:21:39.448043 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a2673ded-91dc-4947-9d06-92a4983afee4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:39.448814 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a2673ded-91dc-4947-9d06-92a4983afee4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:39.455265 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1366/2724] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:21:39 2022] DELETE /identity/v3/projects/0e7c6c045c04466e8a98ff8d02ecd9a7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:39.457204 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ceaead-8916-4994-b036-20c89a881453 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:39.457552 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ceaead-8916-4994-b036-20c89a881453 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:39.457895 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ceaead-8916-4994-b036-20c89a881453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:39.459934 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-30cd7272-3c5a-4d0a-b028-5c66948a7f0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44bb8b34af2240cea047bfbcdcf9e9ab'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:39.482339 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-81ceaead-8916-4994-b036-20c89a881453 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:39.483984 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-81ceaead-8916-4994-b036-20c89a881453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:39.491455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1359/2725] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:21:39 2022] DELETE /identity/v3/users/44bb8b34af2240cea047bfbcdcf9e9ab => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:39.493228 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad6de6-21bb-4242-bedf-f32f1ec819e0 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:39.493831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad6de6-21bb-4242-bedf-f32f1ec819e0 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:39.494114 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad6de6-21bb-4242-bedf-f32f1ec819e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:39.522931 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-baad6de6-21bb-4242-bedf-f32f1ec819e0 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:39.524694 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-baad6de6-21bb-4242-bedf-f32f1ec819e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:39.562639 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1367/2726] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:39.570612 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718cb194-3ab8-42b9-96a8-72ead276bd6e None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:39.570915 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718cb194-3ab8-42b9-96a8-72ead276bd6e None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:39.571196 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718cb194-3ab8-42b9-96a8-72ead276bd6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:39.595554 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1360/2727] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:39.595802 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-718cb194-3ab8-42b9-96a8-72ead276bd6e None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:39.597381 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-718cb194-3ab8-42b9-96a8-72ead276bd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:39.602725 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad6de6-21bb-4242-bedf-f32f1ec819e0 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:39.604931 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-baad6de6-21bb-4242-bedf-f32f1ec819e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:39.606519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1368/2728] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:39.610554 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718cb194-3ab8-42b9-96a8-72ead276bd6e None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:39.622020 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718cb194-3ab8-42b9-96a8-72ead276bd6e None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:39.628679 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a7771-dd88-4a1b-98b9-71331cc9b394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:39.629522 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a7771-dd88-4a1b-98b9-71331cc9b394 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:39.630117 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a7771-dd88-4a1b-98b9-71331cc9b394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:39.630437 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a7771-dd88-4a1b-98b9-71331cc9b394 admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:39.632466 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92a7771-dd88-4a1b-98b9-71331cc9b394 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:21:39.632466 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92a7771-dd88-4a1b-98b9-71331cc9b394 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:39.633657 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92a7771-dd88-4a1b-98b9-71331cc9b394 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:39.637695 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:39.654597 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1369/2729] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:21:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:21:39.655377 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a7771-dd88-4a1b-98b9-71331cc9b394 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:39.667256 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a7771-dd88-4a1b-98b9-71331cc9b394 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:39.673305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe3bba-de83-427f-bfe5-956b218f42e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:39.673919 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe3bba-de83-427f-bfe5-956b218f42e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:39.674203 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe3bba-de83-427f-bfe5-956b218f42e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:39.674478 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe3bba-de83-427f-bfe5-956b218f42e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:39.675177 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe3bba-de83-427f-bfe5-956b218f42e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:39.677069 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe3bba-de83-427f-bfe5-956b218f42e5 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:39.714326 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad6de6-21bb-4242-bedf-f32f1ec819e0 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:39.716312 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1370/2730] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:21:39 2022] POST /identity/v3/projects => generated 377 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:21:39.717207 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe3bba-de83-427f-bfe5-956b218f42e5 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:39.792714 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45724b25-eb04-4772-98fa-18d4bf18f95a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:39.793973 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45724b25-eb04-4772-98fa-18d4bf18f95a service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:39.794304 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45724b25-eb04-4772-98fa-18d4bf18f95a service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:39.794569 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45724b25-eb04-4772-98fa-18d4bf18f95a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:39.812522 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe3bba-de83-427f-bfe5-956b218f42e5 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:39.821533 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069c32d-5f47-474e-a606-760a525562cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:39.822655 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:39.827266 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069c32d-5f47-474e-a606-760a525562cf admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:39.827700 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069c32d-5f47-474e-a606-760a525562cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:39.827889 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069c32d-5f47-474e-a606-760a525562cf admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:39.834750 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069c32d-5f47-474e-a606-760a525562cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:39.834750 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069c32d-5f47-474e-a606-760a525562cf admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:39.862886 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1371/2731] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:21:39 2022] POST /identity/v3/users => generated 289 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:21:39.865033 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069c32d-5f47-474e-a606-760a525562cf admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:39.879069 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069c32d-5f47-474e-a606-760a525562cf admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:39.885914 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9374e05-2ebb-4e18-8655-d39b9fc850fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:39.887410 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9374e05-2ebb-4e18-8655-d39b9fc850fe admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:39.887874 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9374e05-2ebb-4e18-8655-d39b9fc850fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:39.888300 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9374e05-2ebb-4e18-8655-d39b9fc850fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:39.889203 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9374e05-2ebb-4e18-8655-d39b9fc850fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:39.891781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9374e05-2ebb-4e18-8655-d39b9fc850fe admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:39.908979 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1372/2732] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:21:39.909838 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9374e05-2ebb-4e18-8655-d39b9fc850fe admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:39.918993 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45724b25-eb04-4772-98fa-18d4bf18f95a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:39.920798 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45724b25-eb04-4772-98fa-18d4bf18f95a service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:39.926890 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9374e05-2ebb-4e18-8655-d39b9fc850fe admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:39.932893 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d47f653-1b50-4830-9204-2092a457936d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:39.934636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47f653-1b50-4830-9204-2092a457936d admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:39.935534 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47f653-1b50-4830-9204-2092a457936d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:39.935925 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47f653-1b50-4830-9204-2092a457936d admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:39.936842 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d47f653-1b50-4830-9204-2092a457936d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:39.937245 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1361/2733] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:21:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:21:39.940440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d47f653-1b50-4830-9204-2092a457936d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:39.958007 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1373/2734] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:21:39.959605 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45724b25-eb04-4772-98fa-18d4bf18f95a service neutron] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:39.971553 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45724b25-eb04-4772-98fa-18d4bf18f95a service neutron] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:39.976336 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e1412-6ed4-4eaa-9cc6-f3ad02e928ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:39.977010 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e1412-6ed4-4eaa-9cc6-f3ad02e928ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:39.977285 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e1412-6ed4-4eaa-9cc6-f3ad02e928ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:39.977550 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e1412-6ed4-4eaa-9cc6-f3ad02e928ce admin admin] PATH_INFO: `/v3/projects/01cbf4941c6c48b29a860150a980c2fa/users/acc1c8b2d8c54b8b83614f7bf013ddb0/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:40.002872 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24e1412-6ed4-4eaa-9cc6-f3ad02e928ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=01cbf4941c6c48b29a860150a980c2fa, user_id=acc1c8b2d8c54b8b83614f7bf013ddb0, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:40.005751 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24e1412-6ed4-4eaa-9cc6-f3ad02e928ce admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:40.035868 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1362/2735] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:21:39 2022] PUT /identity/v3/projects/01cbf4941c6c48b29a860150a980c2fa/users/acc1c8b2d8c54b8b83614f7bf013ddb0/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:21:40.040109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3981c1d-2c4a-450f-b38d-73af1a3b5769 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:40.040855 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3981c1d-2c4a-450f-b38d-73af1a3b5769 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:40.041373 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3981c1d-2c4a-450f-b38d-73af1a3b5769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:40.077925 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-a3981c1d-2c4a-450f-b38d-73af1a3b5769 None None] MFA Rules not processed for user `acc1c8b2d8c54b8b83614f7bf013ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:40.079986 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-a3981c1d-2c4a-450f-b38d-73af1a3b5769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:40.193040 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1374/2736] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:40.197656 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e1412-6ed4-4eaa-9cc6-f3ad02e928ce admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:40.271289 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3981c1d-2c4a-450f-b38d-73af1a3b5769 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:40.272746 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-a3981c1d-2c4a-450f-b38d-73af1a3b5769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:40.296068 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e1412-6ed4-4eaa-9cc6-f3ad02e928ce admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:40.301284 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f88db-96b7-46df-95ef-cac71f91538b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:40.301978 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f88db-96b7-46df-95ef-cac71f91538b admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:40.302467 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f88db-96b7-46df-95ef-cac71f91538b admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:40.303177 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f88db-96b7-46df-95ef-cac71f91538b admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:40.303573 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67f88db-96b7-46df-95ef-cac71f91538b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:40.305701 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67f88db-96b7-46df-95ef-cac71f91538b admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:40.333546 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:40.336402 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1363/2737] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:21:40 2022] POST /identity/v3/projects => generated 377 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:21:40.341422 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f88db-96b7-46df-95ef-cac71f91538b admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:40.377017 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3981c1d-2c4a-450f-b38d-73af1a3b5769 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:40.423163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f88db-96b7-46df-95ef-cac71f91538b admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:40.428032 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c70a5-2f88-425d-9ea2-62dbc014ef3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:40.428660 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c70a5-2f88-425d-9ea2-62dbc014ef3d admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:40.428964 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c70a5-2f88-425d-9ea2-62dbc014ef3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:40.429231 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c70a5-2f88-425d-9ea2-62dbc014ef3d admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:40.432478 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1c70a5-2f88-425d-9ea2-62dbc014ef3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:40.436313 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1c70a5-2f88-425d-9ea2-62dbc014ef3d admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:40.456086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5adab0-1f09-4111-8860-2fafd2b94fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:40.456747 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5adab0-1f09-4111-8860-2fafd2b94fa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:40.457060 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5adab0-1f09-4111-8860-2fafd2b94fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:40.457620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5adab0-1f09-4111-8860-2fafd2b94fa4 admin admin] PATH_INFO: `/v3/projects/aa0ed12cfa6a44bea6d6894143b8ba71` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:40.459054 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5adab0-1f09-4111-8860-2fafd2b94fa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa0ed12cfa6a44bea6d6894143b8ba71)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:40.461247 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5adab0-1f09-4111-8860-2fafd2b94fa4 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:40.467018 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1364/2738] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:21:40 2022] POST /identity/v3/users => generated 289 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:21:40.469239 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c70a5-2f88-425d-9ea2-62dbc014ef3d admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:40.481899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c70a5-2f88-425d-9ea2-62dbc014ef3d admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:40.486994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d944682e-3b9a-45fc-8980-785f381143fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:40.487620 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944682e-3b9a-45fc-8980-785f381143fe admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:40.487927 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944682e-3b9a-45fc-8980-785f381143fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:40.488193 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944682e-3b9a-45fc-8980-785f381143fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:40.488844 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d944682e-3b9a-45fc-8980-785f381143fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:40.490465 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d944682e-3b9a-45fc-8980-785f381143fe admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:40.491991 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:40.507365 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1365/2739] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:21:40.514883 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d944682e-3b9a-45fc-8980-785f381143fe admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:40.528072 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d944682e-3b9a-45fc-8980-785f381143fe admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:40.539823 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-cb5adab0-1f09-4111-8860-2fafd2b94fa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa0ed12cfa6a44bea6d6894143b8ba71', 'request_id': 'req-cb5adab0-1f09-4111-8860-2fafd2b94fa4'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:40.550107 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-cb5adab0-1f09-4111-8860-2fafd2b94fa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa0ed12cfa6a44bea6d6894143b8ba71', 'request_id': 'req-cb5adab0-1f09-4111-8860-2fafd2b94fa4'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:40.557785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-cb5adab0-1f09-4111-8860-2fafd2b94fa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa0ed12cfa6a44bea6d6894143b8ba71', 'request_id': 'req-cb5adab0-1f09-4111-8860-2fafd2b94fa4'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:40.576500 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-cb5adab0-1f09-4111-8860-2fafd2b94fa4 admin admin] The token cache is being invalidate because project aa0ed12cfa6a44bea6d6894143b8ba71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:21:40.577207 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-cb5adab0-1f09-4111-8860-2fafd2b94fa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:40.577507 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-cb5adab0-1f09-4111-8860-2fafd2b94fa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:40.587004 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1375/2740] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:21:40 2022] DELETE /identity/v3/projects/aa0ed12cfa6a44bea6d6894143b8ba71 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:21:40.587004 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5adab0-1f09-4111-8860-2fafd2b94fa4 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:40.588532 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-cb5adab0-1f09-4111-8860-2fafd2b94fa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:40.614942 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cbd5d8-970e-4614-90d4-e97d10acf648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:40.614942 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbd5d8-970e-4614-90d4-e97d10acf648 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:40.615335 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbd5d8-970e-4614-90d4-e97d10acf648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:40.615335 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbd5d8-970e-4614-90d4-e97d10acf648 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:40.615984 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cbd5d8-970e-4614-90d4-e97d10acf648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:40.617636 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cbd5d8-970e-4614-90d4-e97d10acf648 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:40.632641 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1366/2741] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:21:40.634065 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cbd5d8-970e-4614-90d4-e97d10acf648 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:40.636246 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-b2cbd5d8-970e-4614-90d4-e97d10acf648 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:40.654540 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cbd5d8-970e-4614-90d4-e97d10acf648 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:40.660353 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ccb6c-2f3d-4f69-819a-d47d3d1ebece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:40.661057 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ccb6c-2f3d-4f69-819a-d47d3d1ebece admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:40.661336 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ccb6c-2f3d-4f69-819a-d47d3d1ebece admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:40.662647 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ccb6c-2f3d-4f69-819a-d47d3d1ebece admin admin] PATH_INFO: `/v3/projects/0c207550dc874483aae3067151575d07/users/2348302e3df14e7f98186fea89a9af87/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:40.691211 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ccb6c-2f3d-4f69-819a-d47d3d1ebece admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c207550dc874483aae3067151575d07, user_id=2348302e3df14e7f98186fea89a9af87, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:40.693119 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5adab0-1f09-4111-8860-2fafd2b94fa4 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:40.694115 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ccb6c-2f3d-4f69-819a-d47d3d1ebece admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:40.697961 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d56f1-7e5a-42ce-9417-0168c1b42107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:40.698816 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d56f1-7e5a-42ce-9417-0168c1b42107 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:40.699705 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d56f1-7e5a-42ce-9417-0168c1b42107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:40.699941 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d56f1-7e5a-42ce-9417-0168c1b42107 admin admin] PATH_INFO: `/v3/users/3f2f573644ff40aaab55127b18cb83a6` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:40.701147 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d56f1-7e5a-42ce-9417-0168c1b42107 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f2f573644ff40aaab55127b18cb83a6)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:40.703233 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d56f1-7e5a-42ce-9417-0168c1b42107 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:40.718906 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1367/2742] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:21:40 2022] PUT /identity/v3/projects/0c207550dc874483aae3067151575d07/users/2348302e3df14e7f98186fea89a9af87/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:40.719113 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:40.721963 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c5090-2ae2-466d-b7b6-a282b25d476c None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:40.722250 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c5090-2ae2-466d-b7b6-a282b25d476c None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:40.722973 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c5090-2ae2-466d-b7b6-a282b25d476c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:40.762969 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-e04c5090-2ae2-466d-b7b6-a282b25d476c None None] MFA Rules not processed for user `2348302e3df14e7f98186fea89a9af87`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:40.764664 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-e04c5090-2ae2-466d-b7b6-a282b25d476c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:40.772813 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-157d56f1-7e5a-42ce-9417-0168c1b42107 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f2f573644ff40aaab55127b18cb83a6'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:40.784559 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-157d56f1-7e5a-42ce-9417-0168c1b42107 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f2f573644ff40aaab55127b18cb83a6'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:40.787172 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-157d56f1-7e5a-42ce-9417-0168c1b42107 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f2f573644ff40aaab55127b18cb83a6'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:40.789362 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-157d56f1-7e5a-42ce-9417-0168c1b42107 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f2f573644ff40aaab55127b18cb83a6'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:40.800707 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-157d56f1-7e5a-42ce-9417-0168c1b42107 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f2f573644ff40aaab55127b18cb83a6'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:40.844717 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1376/2743] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:21:40 2022] DELETE /identity/v3/users/3f2f573644ff40aaab55127b18cb83a6 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:21:40.849648 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1368/2744] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:40.850714 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d56f1-7e5a-42ce-9417-0168c1b42107 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:40.852802 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-157d56f1-7e5a-42ce-9417-0168c1b42107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:40.976960 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d56f1-7e5a-42ce-9417-0168c1b42107 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:40.985030 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9fb12-0127-4676-9010-8c087ef8769a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:40.985651 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9fb12-0127-4676-9010-8c087ef8769a admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:40.985934 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9fb12-0127-4676-9010-8c087ef8769a admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:40.986222 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9fb12-0127-4676-9010-8c087ef8769a admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:40.986973 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9fb12-0127-4676-9010-8c087ef8769a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:40.988960 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9fb12-0127-4676-9010-8c087ef8769a admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:41.058286 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c5090-2ae2-466d-b7b6-a282b25d476c None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:41.062933 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1377/2745] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:21:40 2022] POST /identity/v3/projects => generated 375 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:21:41.143219 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:41.152937 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c5090-2ae2-466d-b7b6-a282b25d476c None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:41.205655 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9fb12-0127-4676-9010-8c087ef8769a admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:41.209699 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-3da9fb12-0127-4676-9010-8c087ef8769a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:41.243521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3017a8fb-61b4-4477-bbc2-388f40968127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:41.244139 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017a8fb-61b4-4477-bbc2-388f40968127 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:41.244770 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017a8fb-61b4-4477-bbc2-388f40968127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:41.245046 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017a8fb-61b4-4477-bbc2-388f40968127 admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:41.245678 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3017a8fb-61b4-4477-bbc2-388f40968127 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:41.248220 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3017a8fb-61b4-4477-bbc2-388f40968127 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:41.274884 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1369/2746] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:21:41 2022] POST /identity/v3/users => generated 294 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:21:41.278429 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3017a8fb-61b4-4477-bbc2-388f40968127 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:41.289075 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3017a8fb-61b4-4477-bbc2-388f40968127 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:41.299765 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd13515-afd2-486d-9413-a361883e1dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:41.300391 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd13515-afd2-486d-9413-a361883e1dc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:41.300670 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd13515-afd2-486d-9413-a361883e1dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:41.300945 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd13515-afd2-486d-9413-a361883e1dc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:41.302812 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd13515-afd2-486d-9413-a361883e1dc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:41.304447 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd13515-afd2-486d-9413-a361883e1dc7 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:41.319081 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1370/2747] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:21:41.321060 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd13515-afd2-486d-9413-a361883e1dc7 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:41.325771 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9fb12-0127-4676-9010-8c087ef8769a admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:41.330714 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7054abc-2434-4b32-80ab-9abaef6f7f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:41.331507 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7054abc-2434-4b32-80ab-9abaef6f7f8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:41.331853 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7054abc-2434-4b32-80ab-9abaef6f7f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:41.331965 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd13515-afd2-486d-9413-a361883e1dc7 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:41.332127 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7054abc-2434-4b32-80ab-9abaef6f7f8b admin admin] PATH_INFO: `/v3/projects/4d2c0afcd92a49529027cff8f253f40b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:41.333603 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7054abc-2434-4b32-80ab-9abaef6f7f8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d2c0afcd92a49529027cff8f253f40b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:41.335636 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7054abc-2434-4b32-80ab-9abaef6f7f8b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:41.336448 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d44f93-01f9-41cb-878f-d9c9e5b33d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:41.337285 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d44f93-01f9-41cb-878f-d9c9e5b33d4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:41.337473 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d44f93-01f9-41cb-878f-d9c9e5b33d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:41.337745 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d44f93-01f9-41cb-878f-d9c9e5b33d4c admin admin] PATH_INFO: `/v3/projects/dae1fabb3bd2430494d688094730994a/users/5f717ee157f04aa78d0dd37c978b8f78/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:41.358066 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:41.361634 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d44f93-01f9-41cb-878f-d9c9e5b33d4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dae1fabb3bd2430494d688094730994a, user_id=5f717ee157f04aa78d0dd37c978b8f78, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:41.364518 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d44f93-01f9-41cb-878f-d9c9e5b33d4c admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:41.388780 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1371/2748] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:21:41 2022] PUT /identity/v3/projects/dae1fabb3bd2430494d688094730994a/users/5f717ee157f04aa78d0dd37c978b8f78/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:21:41.390251 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d44f93-01f9-41cb-878f-d9c9e5b33d4c admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:41.418560 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d7054abc-2434-4b32-80ab-9abaef6f7f8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d2c0afcd92a49529027cff8f253f40b', 'request_id': 'req-d7054abc-2434-4b32-80ab-9abaef6f7f8b'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:41.431028 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d7054abc-2434-4b32-80ab-9abaef6f7f8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d2c0afcd92a49529027cff8f253f40b', 'request_id': 'req-d7054abc-2434-4b32-80ab-9abaef6f7f8b'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:41.442506 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d7054abc-2434-4b32-80ab-9abaef6f7f8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d2c0afcd92a49529027cff8f253f40b', 'request_id': 'req-d7054abc-2434-4b32-80ab-9abaef6f7f8b'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:41.458463 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d7054abc-2434-4b32-80ab-9abaef6f7f8b admin admin] The token cache is being invalidate because project 4d2c0afcd92a49529027cff8f253f40b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:21:41.458731 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d7054abc-2434-4b32-80ab-9abaef6f7f8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:41.459437 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d7054abc-2434-4b32-80ab-9abaef6f7f8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:41.464306 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1378/2749] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:21:41 2022] DELETE /identity/v3/projects/4d2c0afcd92a49529027cff8f253f40b => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:41.465985 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd63dc-f133-44da-9c58-e648a1b61da3 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:41.466267 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd63dc-f133-44da-9c58-e648a1b61da3 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:41.466542 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd63dc-f133-44da-9c58-e648a1b61da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:41.495710 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d44f93-01f9-41cb-878f-d9c9e5b33d4c admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:41.502313 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-d7bd63dc-f133-44da-9c58-e648a1b61da3 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:41.503935 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d7bd63dc-f133-44da-9c58-e648a1b61da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:41.588628 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1379/2750] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:41.598240 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ddf51-12af-4dc2-88bf-1a950a0228d8 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:41.598608 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ddf51-12af-4dc2-88bf-1a950a0228d8 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:41.598958 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ddf51-12af-4dc2-88bf-1a950a0228d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:41.602578 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58f36b-7986-41b2-a1ea-53c8ee67182d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:41.603354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58f36b-7986-41b2-a1ea-53c8ee67182d admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:41.603713 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58f36b-7986-41b2-a1ea-53c8ee67182d admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:41.604037 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58f36b-7986-41b2-a1ea-53c8ee67182d admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:41.604849 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c58f36b-7986-41b2-a1ea-53c8ee67182d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:41.607107 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c58f36b-7986-41b2-a1ea-53c8ee67182d admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:41.620400 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1372/2751] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:21:41.624790 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58f36b-7986-41b2-a1ea-53c8ee67182d admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:41.626518 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-8c58f36b-7986-41b2-a1ea-53c8ee67182d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:41.631524 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-636ddf51-12af-4dc2-88bf-1a950a0228d8 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:41.632715 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-636ddf51-12af-4dc2-88bf-1a950a0228d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:41.634451 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:41.642897 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1380/2752] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:41.643210 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58f36b-7986-41b2-a1ea-53c8ee67182d admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:41.648868 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ddf51-12af-4dc2-88bf-1a950a0228d8 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:41.653888 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7264c7-e2ac-4121-966f-0e523c06c0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:41.654604 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7264c7-e2ac-4121-966f-0e523c06c0eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:41.654906 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7264c7-e2ac-4121-966f-0e523c06c0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:41.655186 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7264c7-e2ac-4121-966f-0e523c06c0eb admin admin] PATH_INFO: `/v3/projects/dae1fabb3bd2430494d688094730994a/users/5f717ee157f04aa78d0dd37c978b8f78/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:41.657141 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7264c7-e2ac-4121-966f-0e523c06c0eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=dae1fabb3bd2430494d688094730994a, user_id=5f717ee157f04aa78d0dd37c978b8f78, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:41.659806 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7264c7-e2ac-4121-966f-0e523c06c0eb admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:41.659944 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ddf51-12af-4dc2-88bf-1a950a0228d8 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:41.664670 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d2c97-0f68-4ecb-8d6d-2b46b26342b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:41.665367 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d2c97-0f68-4ecb-8d6d-2b46b26342b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:41.665713 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d2c97-0f68-4ecb-8d6d-2b46b26342b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:41.666048 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d2c97-0f68-4ecb-8d6d-2b46b26342b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:41.668583 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d2c97-0f68-4ecb-8d6d-2b46b26342b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:21:41.668966 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d2c97-0f68-4ecb-8d6d-2b46b26342b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:41.670625 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d2c97-0f68-4ecb-8d6d-2b46b26342b8 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:41.672394 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1373/2753] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:21:41 2022] PUT /identity/v3/projects/dae1fabb3bd2430494d688094730994a/users/5f717ee157f04aa78d0dd37c978b8f78/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:21:41.682870 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ef5dc-4e56-495c-86f7-89f2d121e24f None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:41.683509 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ef5dc-4e56-495c-86f7-89f2d121e24f None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:41.683973 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ef5dc-4e56-495c-86f7-89f2d121e24f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:41.687021 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1381/2754] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:21:41.690859 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d2c97-0f68-4ecb-8d6d-2b46b26342b8 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:41.701391 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d2c97-0f68-4ecb-8d6d-2b46b26342b8 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:41.718424 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-f31ef5dc-4e56-495c-86f7-89f2d121e24f None None] MFA Rules not processed for user `5f717ee157f04aa78d0dd37c978b8f78`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:41.719620 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-f31ef5dc-4e56-495c-86f7-89f2d121e24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:41.776475 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc323944-97bf-41b1-9382-7fa197f40c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:41.777731 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc323944-97bf-41b1-9382-7fa197f40c12 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:41.778092 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc323944-97bf-41b1-9382-7fa197f40c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:41.778430 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc323944-97bf-41b1-9382-7fa197f40c12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:41.779189 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc323944-97bf-41b1-9382-7fa197f40c12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:41.781137 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc323944-97bf-41b1-9382-7fa197f40c12 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:41.808377 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1374/2755] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:41 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:21:41.818605 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1382/2756] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:21:41 2022] POST /identity/v3/projects => generated 379 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:21:41.818806 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31ef5dc-4e56-495c-86f7-89f2d121e24f None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:41.818806 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-f31ef5dc-4e56-495c-86f7-89f2d121e24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:41.824203 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc323944-97bf-41b1-9382-7fa197f40c12 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:41.840142 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:41.927536 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31ef5dc-4e56-495c-86f7-89f2d121e24f None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:41.934591 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc323944-97bf-41b1-9382-7fa197f40c12 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:41.940972 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a33a3e-6f25-4770-ae03-261cd7233642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:41.941917 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a33a3e-6f25-4770-ae03-261cd7233642 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:41.942197 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a33a3e-6f25-4770-ae03-261cd7233642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:41.942472 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a33a3e-6f25-4770-ae03-261cd7233642 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:41.943220 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a33a3e-6f25-4770-ae03-261cd7233642 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:41.945387 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a33a3e-6f25-4770-ae03-261cd7233642 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:41.989904 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1383/2757] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:21:41 2022] POST /identity/v3/users => generated 290 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:21:41.991006 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a33a3e-6f25-4770-ae03-261cd7233642 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:42.006670 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a33a3e-6f25-4770-ae03-261cd7233642 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:42.007575 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26667e82-6aaa-4ed3-bc42-5af67cbaea53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:42.008416 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26667e82-6aaa-4ed3-bc42-5af67cbaea53 service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:42.008770 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26667e82-6aaa-4ed3-bc42-5af67cbaea53 service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:42.009102 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26667e82-6aaa-4ed3-bc42-5af67cbaea53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:42.011379 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-26667e82-6aaa-4ed3-bc42-5af67cbaea53 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:42.012668 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae785f9e-dbb2-4f0e-8103-c7dc4eed89ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:42.013284 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae785f9e-dbb2-4f0e-8103-c7dc4eed89ee admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:42.013564 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae785f9e-dbb2-4f0e-8103-c7dc4eed89ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:42.013937 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae785f9e-dbb2-4f0e-8103-c7dc4eed89ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:42.014735 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae785f9e-dbb2-4f0e-8103-c7dc4eed89ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:42.016455 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae785f9e-dbb2-4f0e-8103-c7dc4eed89ee admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:42.033821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1384/2758] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:21:42.036002 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae785f9e-dbb2-4f0e-8103-c7dc4eed89ee admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:42.045537 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:42.048662 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae785f9e-dbb2-4f0e-8103-c7dc4eed89ee admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:42.053888 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e31131-5e7f-4a38-977a-0a649f83cb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:42.057080 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e31131-5e7f-4a38-977a-0a649f83cb9f admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:42.057369 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e31131-5e7f-4a38-977a-0a649f83cb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:42.057641 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e31131-5e7f-4a38-977a-0a649f83cb9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:42.058338 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e31131-5e7f-4a38-977a-0a649f83cb9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:42.061595 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e31131-5e7f-4a38-977a-0a649f83cb9f admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:42.078889 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1385/2759] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:21:42.081787 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e31131-5e7f-4a38-977a-0a649f83cb9f admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:42.095340 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e31131-5e7f-4a38-977a-0a649f83cb9f admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:42.095917 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26667e82-6aaa-4ed3-bc42-5af67cbaea53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:42.097360 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26667e82-6aaa-4ed3-bc42-5af67cbaea53 service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:42.102063 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6371fe7-58f8-4f5a-bf90-6f04f8eaf2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:42.103061 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6371fe7-58f8-4f5a-bf90-6f04f8eaf2f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:42.103548 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6371fe7-58f8-4f5a-bf90-6f04f8eaf2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:42.104037 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6371fe7-58f8-4f5a-bf90-6f04f8eaf2f1 admin admin] PATH_INFO: `/v3/projects/238807e2a18d423da310a7d679e9d153/users/dbf87883bbf54d70a707c02cc9b8a473/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:42.111039 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1375/2760] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:21:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:21:42.122686 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6371fe7-58f8-4f5a-bf90-6f04f8eaf2f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=238807e2a18d423da310a7d679e9d153, user_id=dbf87883bbf54d70a707c02cc9b8a473, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:42.127342 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6371fe7-58f8-4f5a-bf90-6f04f8eaf2f1 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:42.148802 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1386/2761] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:21:42 2022] PUT /identity/v3/projects/238807e2a18d423da310a7d679e9d153/users/dbf87883bbf54d70a707c02cc9b8a473/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:21:42.160462 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c01aa-a5a8-4019-a0e6-f414018c64c0 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:42.160773 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c01aa-a5a8-4019-a0e6-f414018c64c0 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:42.161013 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c01aa-a5a8-4019-a0e6-f414018c64c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:42.194935 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-826c01aa-a5a8-4019-a0e6-f414018c64c0 None None] MFA Rules not processed for user `dbf87883bbf54d70a707c02cc9b8a473`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:42.196170 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-826c01aa-a5a8-4019-a0e6-f414018c64c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:42.283270 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1376/2762] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:42.285448 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6371fe7-58f8-4f5a-bf90-6f04f8eaf2f1 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:42.373076 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6371fe7-58f8-4f5a-bf90-6f04f8eaf2f1 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:42.378139 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25b4e71-0005-41b0-a37f-5ae836587627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:42.378785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b4e71-0005-41b0-a37f-5ae836587627 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:42.379198 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b4e71-0005-41b0-a37f-5ae836587627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:42.379509 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b4e71-0005-41b0-a37f-5ae836587627 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:42.380206 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25b4e71-0005-41b0-a37f-5ae836587627 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:42.382534 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25b4e71-0005-41b0-a37f-5ae836587627 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:42.424814 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1387/2763] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:21:42 2022] POST /identity/v3/projects => generated 375 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:21:42.428744 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c01aa-a5a8-4019-a0e6-f414018c64c0 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:42.444538 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c01aa-a5a8-4019-a0e6-f414018c64c0 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:42.549837 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c9792e-2415-4473-b638-6486fe49c436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:42.550465 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9792e-2415-4473-b638-6486fe49c436 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:42.550741 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9792e-2415-4473-b638-6486fe49c436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:42.551059 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9792e-2415-4473-b638-6486fe49c436 admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:42.551725 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c9792e-2415-4473-b638-6486fe49c436 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:42.553629 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c9792e-2415-4473-b638-6486fe49c436 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:42.580652 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1377/2764] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:21:42 2022] POST /identity/v3/users => generated 288 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:21:42.584361 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25b4e71-0005-41b0-a37f-5ae836587627 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:42.596711 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25b4e71-0005-41b0-a37f-5ae836587627 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:42.602067 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e4e34c-a4c4-4ca5-8416-eb6bd874d508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:42.602492 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4e34c-a4c4-4ca5-8416-eb6bd874d508 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:42.602805 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4e34c-a4c4-4ca5-8416-eb6bd874d508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:42.603101 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4e34c-a4c4-4ca5-8416-eb6bd874d508 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:42.603981 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e4e34c-a4c4-4ca5-8416-eb6bd874d508 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:42.607133 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e4e34c-a4c4-4ca5-8416-eb6bd874d508 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:42.619375 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1388/2765] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:21:42.623307 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c9792e-2415-4473-b638-6486fe49c436 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:42.634102 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c9792e-2415-4473-b638-6486fe49c436 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:42.643211 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c394202-90c0-4779-ba0b-bb9b6f83229a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:42.644167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c394202-90c0-4779-ba0b-bb9b6f83229a admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:42.644609 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c394202-90c0-4779-ba0b-bb9b6f83229a admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:42.644982 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c394202-90c0-4779-ba0b-bb9b6f83229a admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:42.645994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c394202-90c0-4779-ba0b-bb9b6f83229a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:42.647725 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c394202-90c0-4779-ba0b-bb9b6f83229a admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:42.660755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1378/2766] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:21:42.664615 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e4e34c-a4c4-4ca5-8416-eb6bd874d508 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:42.675517 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e4e34c-a4c4-4ca5-8416-eb6bd874d508 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:42.682797 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8b841-19e7-4d70-a8ae-ea53b2e00a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:42.683847 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8b841-19e7-4d70-a8ae-ea53b2e00a74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:42.684356 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8b841-19e7-4d70-a8ae-ea53b2e00a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:42.685107 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8b841-19e7-4d70-a8ae-ea53b2e00a74 admin admin] PATH_INFO: `/v3/projects/8954e46202cd46b09accb6fe9e32c637/users/c8bfff96a553443da92cf513b9d21efe/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:42.702182 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8b841-19e7-4d70-a8ae-ea53b2e00a74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8954e46202cd46b09accb6fe9e32c637, user_id=c8bfff96a553443da92cf513b9d21efe, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:42.704907 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8b841-19e7-4d70-a8ae-ea53b2e00a74 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:42.718872 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1389/2767] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:21:42 2022] PUT /identity/v3/projects/8954e46202cd46b09accb6fe9e32c637/users/c8bfff96a553443da92cf513b9d21efe/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:42.787109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0faa41-0f8f-4532-967d-ed0147a42c98 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:42.787540 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0faa41-0f8f-4532-967d-ed0147a42c98 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:42.787653 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0faa41-0f8f-4532-967d-ed0147a42c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:42.817708 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-ad0faa41-0f8f-4532-967d-ed0147a42c98 None None] MFA Rules not processed for user `c8bfff96a553443da92cf513b9d21efe`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:42.819360 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-ad0faa41-0f8f-4532-967d-ed0147a42c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:42.844945 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:42.898896 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1379/2768] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:42.904337 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8b841-19e7-4d70-a8ae-ea53b2e00a74 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:42.909212 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:42.976610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8b841-19e7-4d70-a8ae-ea53b2e00a74 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:42.982357 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49da465-5913-40a1-9098-04d4538d63c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:42.983201 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49da465-5913-40a1-9098-04d4538d63c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:42.983575 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49da465-5913-40a1-9098-04d4538d63c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:42.983923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49da465-5913-40a1-9098-04d4538d63c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:42.984642 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49da465-5913-40a1-9098-04d4538d63c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:42.986596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49da465-5913-40a1-9098-04d4538d63c2 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:43.024594 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1390/2769] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:21:42 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:21:43.026600 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0faa41-0f8f-4532-967d-ed0147a42c98 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:43.042602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0faa41-0f8f-4532-967d-ed0147a42c98 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:43.118137 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc30c59-82fd-4cfc-9ed4-de111927dba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:43.118758 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc30c59-82fd-4cfc-9ed4-de111927dba2 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:43.119042 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc30c59-82fd-4cfc-9ed4-de111927dba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:43.119365 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc30c59-82fd-4cfc-9ed4-de111927dba2 admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:43.120116 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc30c59-82fd-4cfc-9ed4-de111927dba2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:43.122383 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc30c59-82fd-4cfc-9ed4-de111927dba2 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:43.146100 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1380/2770] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:21:43 2022] POST /identity/v3/users => generated 295 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:21:43.152519 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49da465-5913-40a1-9098-04d4538d63c2 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:43.165786 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49da465-5913-40a1-9098-04d4538d63c2 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:43.171540 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24041c00-0569-4cfe-b24d-e84a12e9137e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:43.172577 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24041c00-0569-4cfe-b24d-e84a12e9137e admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:43.172966 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24041c00-0569-4cfe-b24d-e84a12e9137e admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:43.173406 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24041c00-0569-4cfe-b24d-e84a12e9137e admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:43.174261 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24041c00-0569-4cfe-b24d-e84a12e9137e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:43.175989 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24041c00-0569-4cfe-b24d-e84a12e9137e admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:43.189368 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1391/2771] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:21:43.193667 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc30c59-82fd-4cfc-9ed4-de111927dba2 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:43.206163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc30c59-82fd-4cfc-9ed4-de111927dba2 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:43.211148 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42914871-e943-4652-8322-f6b43f2d29bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:43.212005 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42914871-e943-4652-8322-f6b43f2d29bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:43.212558 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42914871-e943-4652-8322-f6b43f2d29bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:43.213160 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42914871-e943-4652-8322-f6b43f2d29bd admin admin] PATH_INFO: `/v3/projects/2fc77d8af99442f1aea9f0e2f0c882b8/users/231ea1e1c9f9451080d18ad29f03aaca/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:43.230222 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42914871-e943-4652-8322-f6b43f2d29bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fc77d8af99442f1aea9f0e2f0c882b8, user_id=231ea1e1c9f9451080d18ad29f03aaca, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:43.233178 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42914871-e943-4652-8322-f6b43f2d29bd admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:43.248781 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1381/2772] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:21:43 2022] PUT /identity/v3/projects/2fc77d8af99442f1aea9f0e2f0c882b8/users/231ea1e1c9f9451080d18ad29f03aaca/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:21:43.252688 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24041c00-0569-4cfe-b24d-e84a12e9137e admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:43.264057 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24041c00-0569-4cfe-b24d-e84a12e9137e admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:43.326771 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a45b4-e14f-4c44-b942-8b7d4431bce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:43.327454 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a45b4-e14f-4c44-b942-8b7d4431bce4 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:43.327735 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a45b4-e14f-4c44-b942-8b7d4431bce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:43.328041 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a45b4-e14f-4c44-b942-8b7d4431bce4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:43.328690 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a45b4-e14f-4c44-b942-8b7d4431bce4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:43.330312 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a45b4-e14f-4c44-b942-8b7d4431bce4 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:43.346061 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42914871-e943-4652-8322-f6b43f2d29bd admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:43.347165 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1392/2773] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:21:43.359084 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42914871-e943-4652-8322-f6b43f2d29bd admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:43.364183 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d51cf-28a6-47ee-bbd8-f658b0ecb9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:43.364907 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d51cf-28a6-47ee-bbd8-f658b0ecb9ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:43.365185 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d51cf-28a6-47ee-bbd8-f658b0ecb9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:43.365524 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d51cf-28a6-47ee-bbd8-f658b0ecb9ab admin admin] PATH_INFO: `/v3/projects/2fc77d8af99442f1aea9f0e2f0c882b8/users/231ea1e1c9f9451080d18ad29f03aaca/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:43.368157 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d51cf-28a6-47ee-bbd8-f658b0ecb9ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fc77d8af99442f1aea9f0e2f0c882b8, user_id=231ea1e1c9f9451080d18ad29f03aaca, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:43.370997 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d51cf-28a6-47ee-bbd8-f658b0ecb9ab admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:43.383735 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1382/2774] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:21:43 2022] PUT /identity/v3/projects/2fc77d8af99442f1aea9f0e2f0c882b8/users/231ea1e1c9f9451080d18ad29f03aaca/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:21:43.396996 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c89151-45d2-4b87-893b-8afe71d4c3cf None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:43.397293 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c89151-45d2-4b87-893b-8afe71d4c3cf None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:43.397570 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c89151-45d2-4b87-893b-8afe71d4c3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:43.432797 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-59c89151-45d2-4b87-893b-8afe71d4c3cf None None] MFA Rules not processed for user `231ea1e1c9f9451080d18ad29f03aaca`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:43.434151 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-59c89151-45d2-4b87-893b-8afe71d4c3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:43.511281 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:43.524237 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1393/2775] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:43 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:21:43.531359 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d51cf-28a6-47ee-bbd8-f658b0ecb9ab admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:43.644149 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d51cf-28a6-47ee-bbd8-f658b0ecb9ab admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:43.648765 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcde768-2280-4757-a0b5-e5861afa2f2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:43.649342 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcde768-2280-4757-a0b5-e5861afa2f2f service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:43.650304 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcde768-2280-4757-a0b5-e5861afa2f2f service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:43.650657 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcde768-2280-4757-a0b5-e5861afa2f2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:43.662664 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcde768-2280-4757-a0b5-e5861afa2f2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:43.664176 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcde768-2280-4757-a0b5-e5861afa2f2f service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:43.676235 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1383/2776] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:21:43 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:21:43.728008 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c89151-45d2-4b87-893b-8afe71d4c3cf None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:43.747207 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c89151-45d2-4b87-893b-8afe71d4c3cf None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:43.751755 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d1fc3b-01d3-45b9-8519-93b6ddcaecaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:43.754067 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1fc3b-01d3-45b9-8519-93b6ddcaecaa service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:43.754067 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1fc3b-01d3-45b9-8519-93b6ddcaecaa service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:43.754067 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1fc3b-01d3-45b9-8519-93b6ddcaecaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:43.803413 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcde768-2280-4757-a0b5-e5861afa2f2f service cinder] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:43.804867 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-1fcde768-2280-4757-a0b5-e5861afa2f2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:43.818061 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcde768-2280-4757-a0b5-e5861afa2f2f service cinder] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:43.821770 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e29ce-498b-4817-89f0-ad592df635a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:43.822371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e29ce-498b-4817-89f0-ad592df635a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:43.822641 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e29ce-498b-4817-89f0-ad592df635a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:43.822937 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e29ce-498b-4817-89f0-ad592df635a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:43.825677 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-945e29ce-498b-4817-89f0-ad592df635a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:43.843787 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:43.861540 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d1fc3b-01d3-45b9-8519-93b6ddcaecaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:43.863197 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d1fc3b-01d3-45b9-8519-93b6ddcaecaa service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:43.874596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1394/2777] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:21:43 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:21:43.925372 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e29ce-498b-4817-89f0-ad592df635a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:43.926798 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e29ce-498b-4817-89f0-ad592df635a1 service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:43.938871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1384/2778] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:21:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:21:44.860839 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d1fc3b-01d3-45b9-8519-93b6ddcaecaa service cinder] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:44.869855 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d1fc3b-01d3-45b9-8519-93b6ddcaecaa service cinder] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:44.873647 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98702e2-b0b2-4836-bfd6-ba9610ddfb90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:44.874250 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98702e2-b0b2-4836-bfd6-ba9610ddfb90 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:44.874525 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98702e2-b0b2-4836-bfd6-ba9610ddfb90 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:44.874818 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98702e2-b0b2-4836-bfd6-ba9610ddfb90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:44.943171 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98702e2-b0b2-4836-bfd6-ba9610ddfb90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:44.944514 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98702e2-b0b2-4836-bfd6-ba9610ddfb90 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:44.956144 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1395/2779] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:21:44 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:21:46.973083 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e29ce-498b-4817-89f0-ad592df635a1 service cinder] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:47.053151 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e29ce-498b-4817-89f0-ad592df635a1 service cinder] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:47.058676 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2714d6-9326-4cc3-9924-a7334db857fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:47.059620 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2714d6-9326-4cc3-9924-a7334db857fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:47.060420 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2714d6-9326-4cc3-9924-a7334db857fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:47.060694 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2714d6-9326-4cc3-9924-a7334db857fd admin admin] PATH_INFO: `/v3/users/dbf87883bbf54d70a707c02cc9b8a473` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:47.061871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2714d6-9326-4cc3-9924-a7334db857fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbf87883bbf54d70a707c02cc9b8a473)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:47.063929 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2714d6-9326-4cc3-9924-a7334db857fd admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:47.129425 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-8b2714d6-9326-4cc3-9924-a7334db857fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbf87883bbf54d70a707c02cc9b8a473'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:47.146792 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-8b2714d6-9326-4cc3-9924-a7334db857fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbf87883bbf54d70a707c02cc9b8a473'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:47.148032 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-8b2714d6-9326-4cc3-9924-a7334db857fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbf87883bbf54d70a707c02cc9b8a473'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:47.148995 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-8b2714d6-9326-4cc3-9924-a7334db857fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbf87883bbf54d70a707c02cc9b8a473'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:47.158026 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-8b2714d6-9326-4cc3-9924-a7334db857fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbf87883bbf54d70a707c02cc9b8a473'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:47.190915 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1385/2780] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:21:46 2022] DELETE /identity/v3/users/dbf87883bbf54d70a707c02cc9b8a473 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:47.194273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4cd03-d42e-4b14-b3b4-a6c4cd02811b None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:47.195054 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4cd03-d42e-4b14-b3b4-a6c4cd02811b None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:47.195054 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4cd03-d42e-4b14-b3b4-a6c4cd02811b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:47.225533 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-8db4cd03-d42e-4b14-b3b4-a6c4cd02811b None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:47.226914 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-8db4cd03-d42e-4b14-b3b4-a6c4cd02811b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:47.310022 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1396/2781] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:47.319819 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2714d6-9326-4cc3-9924-a7334db857fd admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:47.321371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-8b2714d6-9326-4cc3-9924-a7334db857fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:47.363944 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:47.386113 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2714d6-9326-4cc3-9924-a7334db857fd admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:47.394878 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82adac1-0ab4-4637-b6db-477fdb99f88b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:47.394878 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82adac1-0ab4-4637-b6db-477fdb99f88b service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:47.394878 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82adac1-0ab4-4637-b6db-477fdb99f88b service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:47.394878 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82adac1-0ab4-4637-b6db-477fdb99f88b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:47.402392 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82adac1-0ab4-4637-b6db-477fdb99f88b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:47.403918 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82adac1-0ab4-4637-b6db-477fdb99f88b service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:47.414884 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1386/2782] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:21:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:21:47.732593 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4cd03-d42e-4b14-b3b4-a6c4cd02811b None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:47.734008 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-8db4cd03-d42e-4b14-b3b4-a6c4cd02811b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:47.745939 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4cd03-d42e-4b14-b3b4-a6c4cd02811b None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:47.754333 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc3d77f-c922-468b-8949-48f8673a421a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:47.755065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3d77f-c922-468b-8949-48f8673a421a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:47.755348 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3d77f-c922-468b-8949-48f8673a421a admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:47.755620 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3d77f-c922-468b-8949-48f8673a421a admin admin] PATH_INFO: `/v3/projects/238807e2a18d423da310a7d679e9d153` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:47.757136 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc3d77f-c922-468b-8949-48f8673a421a admin admin] RBAC: Authorizing `identity:delete_project(project_id=238807e2a18d423da310a7d679e9d153)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:47.759093 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc3d77f-c922-468b-8949-48f8673a421a admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:47.794950 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:47.812084 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-dcc3d77f-c922-468b-8949-48f8673a421a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '238807e2a18d423da310a7d679e9d153', 'request_id': 'req-dcc3d77f-c922-468b-8949-48f8673a421a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:47.819713 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-dcc3d77f-c922-468b-8949-48f8673a421a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '238807e2a18d423da310a7d679e9d153', 'request_id': 'req-dcc3d77f-c922-468b-8949-48f8673a421a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:47.826638 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-dcc3d77f-c922-468b-8949-48f8673a421a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '238807e2a18d423da310a7d679e9d153', 'request_id': 'req-dcc3d77f-c922-468b-8949-48f8673a421a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:47.840028 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-dcc3d77f-c922-468b-8949-48f8673a421a admin admin] The token cache is being invalidate because project 238807e2a18d423da310a7d679e9d153 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:21:47.840413 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-dcc3d77f-c922-468b-8949-48f8673a421a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:47.841134 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-dcc3d77f-c922-468b-8949-48f8673a421a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:47.842675 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1397/2783] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:21:47 2022] DELETE /identity/v3/projects/238807e2a18d423da310a7d679e9d153 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:47.846865 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82adac1-0ab4-4637-b6db-477fdb99f88b service neutron] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:47.857331 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82adac1-0ab4-4637-b6db-477fdb99f88b service neutron] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:47.919448 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f79206-55a1-418b-878a-34f3ab6b8ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:47.920139 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f79206-55a1-418b-878a-34f3ab6b8ec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:47.920607 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f79206-55a1-418b-878a-34f3ab6b8ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:47.920934 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f79206-55a1-418b-878a-34f3ab6b8ec9 admin admin] PATH_INFO: `/v3/users/c8bfff96a553443da92cf513b9d21efe` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:47.922094 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f79206-55a1-418b-878a-34f3ab6b8ec9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8bfff96a553443da92cf513b9d21efe)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:47.924008 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f79206-55a1-418b-878a-34f3ab6b8ec9 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:47.969054 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-65f79206-55a1-418b-878a-34f3ab6b8ec9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8bfff96a553443da92cf513b9d21efe'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:47.975937 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-65f79206-55a1-418b-878a-34f3ab6b8ec9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8bfff96a553443da92cf513b9d21efe'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:47.976560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-65f79206-55a1-418b-878a-34f3ab6b8ec9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8bfff96a553443da92cf513b9d21efe'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:47.977163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-65f79206-55a1-418b-878a-34f3ab6b8ec9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8bfff96a553443da92cf513b9d21efe'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:47.984092 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-65f79206-55a1-418b-878a-34f3ab6b8ec9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8bfff96a553443da92cf513b9d21efe'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:48.001421 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1387/2784] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:21:47 2022] DELETE /identity/v3/users/c8bfff96a553443da92cf513b9d21efe => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:48.279201 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de4f09-0760-494f-a448-b2afba5d69c4 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:48.279821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de4f09-0760-494f-a448-b2afba5d69c4 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:48.279821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de4f09-0760-494f-a448-b2afba5d69c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:48.293676 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f79206-55a1-418b-878a-34f3ab6b8ec9 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:48.295565 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-65f79206-55a1-418b-878a-34f3ab6b8ec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:48.302168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-d1de4f09-0760-494f-a448-b2afba5d69c4 None None] MFA Rules not processed for user `1405e758a3af4e8ba659f498cd28ace1`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:48.303764 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d1de4f09-0760-494f-a448-b2afba5d69c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:48.359344 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:48.363597 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f79206-55a1-418b-878a-34f3ab6b8ec9 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:48.364789 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1398/2785] 10.222.0.42 () {58 vars in 967 bytes} [Mon Sep 12 14:21:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:48.370005 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d60f5c-f0a3-48c1-9d9a-7d538ea7cc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:48.371211 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d60f5c-f0a3-48c1-9d9a-7d538ea7cc12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:48.371885 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d60f5c-f0a3-48c1-9d9a-7d538ea7cc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:48.372267 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d60f5c-f0a3-48c1-9d9a-7d538ea7cc12 admin admin] PATH_INFO: `/v3/projects/8954e46202cd46b09accb6fe9e32c637` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:48.373703 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d60f5c-f0a3-48c1-9d9a-7d538ea7cc12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8954e46202cd46b09accb6fe9e32c637)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:48.376328 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d60f5c-f0a3-48c1-9d9a-7d538ea7cc12 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:48.378414 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de4f09-0760-494f-a448-b2afba5d69c4 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:48.379830 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d1de4f09-0760-494f-a448-b2afba5d69c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:48.424254 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-d5d60f5c-f0a3-48c1-9d9a-7d538ea7cc12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8954e46202cd46b09accb6fe9e32c637', 'request_id': 'req-d5d60f5c-f0a3-48c1-9d9a-7d538ea7cc12'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:48.430080 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-d5d60f5c-f0a3-48c1-9d9a-7d538ea7cc12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8954e46202cd46b09accb6fe9e32c637', 'request_id': 'req-d5d60f5c-f0a3-48c1-9d9a-7d538ea7cc12'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:48.437639 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-d5d60f5c-f0a3-48c1-9d9a-7d538ea7cc12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8954e46202cd46b09accb6fe9e32c637', 'request_id': 'req-d5d60f5c-f0a3-48c1-9d9a-7d538ea7cc12'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:48.450848 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-d5d60f5c-f0a3-48c1-9d9a-7d538ea7cc12 admin admin] The token cache is being invalidate because project 8954e46202cd46b09accb6fe9e32c637 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:21:48.451162 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-d5d60f5c-f0a3-48c1-9d9a-7d538ea7cc12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:48.451791 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-d5d60f5c-f0a3-48c1-9d9a-7d538ea7cc12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:48.453943 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1388/2786] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:21:48 2022] DELETE /identity/v3/projects/8954e46202cd46b09accb6fe9e32c637 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:21:48.457114 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d60f5c-f0a3-48c1-9d9a-7d538ea7cc12 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:48.458870 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de4f09-0760-494f-a448-b2afba5d69c4 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:48.459050 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-d5d60f5c-f0a3-48c1-9d9a-7d538ea7cc12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:48.512958 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:48.520567 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae99a4-1343-453b-b197-e537910e5868 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a87ea9c3c5409b97a35c4857ae3ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:48.521304 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae99a4-1343-453b-b197-e537910e5868 service swift] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:48.522004 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae99a4-1343-453b-b197-e537910e5868 service swift] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:48.522311 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae99a4-1343-453b-b197-e537910e5868 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:48.524902 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-66ae99a4-1343-453b-b197-e537910e5868 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:48.528095 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d60f5c-f0a3-48c1-9d9a-7d538ea7cc12 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:48.532858 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e171b-9ce6-4c8b-a43b-478b72b23cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:48.533487 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e171b-9ce6-4c8b-a43b-478b72b23cab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:48.533768 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e171b-9ce6-4c8b-a43b-478b72b23cab admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:48.534049 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e171b-9ce6-4c8b-a43b-478b72b23cab admin admin] PATH_INFO: `/v3/users/231ea1e1c9f9451080d18ad29f03aaca` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:48.535191 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e171b-9ce6-4c8b-a43b-478b72b23cab admin admin] RBAC: Authorizing `identity:delete_user(user_id=231ea1e1c9f9451080d18ad29f03aaca)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:48.537092 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e171b-9ce6-4c8b-a43b-478b72b23cab admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:48.586376 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae99a4-1343-453b-b197-e537910e5868 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:48.588048 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae99a4-1343-453b-b197-e537910e5868 service swift] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:48.588299 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-638e171b-9ce6-4c8b-a43b-478b72b23cab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '231ea1e1c9f9451080d18ad29f03aaca'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:48.595562 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-638e171b-9ce6-4c8b-a43b-478b72b23cab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '231ea1e1c9f9451080d18ad29f03aaca'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:48.596180 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-638e171b-9ce6-4c8b-a43b-478b72b23cab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '231ea1e1c9f9451080d18ad29f03aaca'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:48.597042 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1399/2787] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Sep 12 14:21:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 12 14:21:48.597165 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-638e171b-9ce6-4c8b-a43b-478b72b23cab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '231ea1e1c9f9451080d18ad29f03aaca'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:48.604266 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-638e171b-9ce6-4c8b-a43b-478b72b23cab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '231ea1e1c9f9451080d18ad29f03aaca'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:48.626761 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1389/2788] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:21:48 2022] DELETE /identity/v3/users/231ea1e1c9f9451080d18ad29f03aaca => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:21:48.926315 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae99a4-1343-453b-b197-e537910e5868 service swift] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:48.944396 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae99a4-1343-453b-b197-e537910e5868 service swift] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:48.990981 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e171b-9ce6-4c8b-a43b-478b72b23cab admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:48.992623 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-638e171b-9ce6-4c8b-a43b-478b72b23cab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:49.004969 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:49.022063 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a6f2a9-226e-449c-9fa3-dbfb426b4ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:49.022742 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6f2a9-226e-449c-9fa3-dbfb426b4ff0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:49.023134 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6f2a9-226e-449c-9fa3-dbfb426b4ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:49.023407 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6f2a9-226e-449c-9fa3-dbfb426b4ff0 admin admin] PATH_INFO: `/v3/projects/2fc77d8af99442f1aea9f0e2f0c882b8` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:49.024724 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a6f2a9-226e-449c-9fa3-dbfb426b4ff0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fc77d8af99442f1aea9f0e2f0c882b8)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:49.026721 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a6f2a9-226e-449c-9fa3-dbfb426b4ff0 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:49.071493 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e171b-9ce6-4c8b-a43b-478b72b23cab admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:49.075489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b7a6f2a9-226e-449c-9fa3-dbfb426b4ff0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fc77d8af99442f1aea9f0e2f0c882b8', 'request_id': 'req-b7a6f2a9-226e-449c-9fa3-dbfb426b4ff0'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:49.083786 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b7a6f2a9-226e-449c-9fa3-dbfb426b4ff0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fc77d8af99442f1aea9f0e2f0c882b8', 'request_id': 'req-b7a6f2a9-226e-449c-9fa3-dbfb426b4ff0'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:49.090187 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b7a6f2a9-226e-449c-9fa3-dbfb426b4ff0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fc77d8af99442f1aea9f0e2f0c882b8', 'request_id': 'req-b7a6f2a9-226e-449c-9fa3-dbfb426b4ff0'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:49.102051 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b7a6f2a9-226e-449c-9fa3-dbfb426b4ff0 admin admin] The token cache is being invalidate because project 2fc77d8af99442f1aea9f0e2f0c882b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:21:49.102382 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b7a6f2a9-226e-449c-9fa3-dbfb426b4ff0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:49.103077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b7a6f2a9-226e-449c-9fa3-dbfb426b4ff0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:49.104634 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1400/2789] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:21:48 2022] DELETE /identity/v3/projects/2fc77d8af99442f1aea9f0e2f0c882b8 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:49.111101 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b5250-8924-49f2-8b0e-574826b5be98 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:49.111389 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b5250-8924-49f2-8b0e-574826b5be98 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:49.111656 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b5250-8924-49f2-8b0e-574826b5be98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:49.128625 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c97b4df-5b83-49d6-aa18-21e97ea0c790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:49.129252 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97b4df-5b83-49d6-aa18-21e97ea0c790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:49.129526 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97b4df-5b83-49d6-aa18-21e97ea0c790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:49.129794 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97b4df-5b83-49d6-aa18-21e97ea0c790 admin admin] PATH_INFO: `/v3/users/acc1c8b2d8c54b8b83614f7bf013ddb0` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:49.130950 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c97b4df-5b83-49d6-aa18-21e97ea0c790 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acc1c8b2d8c54b8b83614f7bf013ddb0)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:49.131459 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-7a8b5250-8924-49f2-8b0e-574826b5be98 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:49.132784 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7a8b5250-8924-49f2-8b0e-574826b5be98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:49.133099 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c97b4df-5b83-49d6-aa18-21e97ea0c790 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:49.142217 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1401/2790] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:21:49.152798 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419ef0b-87b0-483f-a1e7-1df44f367527 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:49.153079 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419ef0b-87b0-483f-a1e7-1df44f367527 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:49.153476 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419ef0b-87b0-483f-a1e7-1df44f367527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:49.176011 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-b419ef0b-87b0-483f-a1e7-1df44f367527 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:49.177265 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b419ef0b-87b0-483f-a1e7-1df44f367527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:49.183645 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-4c97b4df-5b83-49d6-aa18-21e97ea0c790 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acc1c8b2d8c54b8b83614f7bf013ddb0'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:49.185997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1402/2791] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:21:49.189995 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b419ef0b-87b0-483f-a1e7-1df44f367527 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:49.192501 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-4c97b4df-5b83-49d6-aa18-21e97ea0c790 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acc1c8b2d8c54b8b83614f7bf013ddb0'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:49.193208 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-4c97b4df-5b83-49d6-aa18-21e97ea0c790 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acc1c8b2d8c54b8b83614f7bf013ddb0'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:49.193819 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-4c97b4df-5b83-49d6-aa18-21e97ea0c790 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acc1c8b2d8c54b8b83614f7bf013ddb0'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:49.200467 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b419ef0b-87b0-483f-a1e7-1df44f367527 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:49.201356 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-4c97b4df-5b83-49d6-aa18-21e97ea0c790 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acc1c8b2d8c54b8b83614f7bf013ddb0'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:49.205855 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eeda72-5d23-4da1-b033-0442edf07b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:49.206603 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eeda72-5d23-4da1-b033-0442edf07b73 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:49.206932 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eeda72-5d23-4da1-b033-0442edf07b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:49.207261 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eeda72-5d23-4da1-b033-0442edf07b73 admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:49.207975 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eeda72-5d23-4da1-b033-0442edf07b73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:21:49.208215 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eeda72-5d23-4da1-b033-0442edf07b73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:49.210154 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eeda72-5d23-4da1-b033-0442edf07b73 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:49.221505 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1390/2792] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:21:48 2022] DELETE /identity/v3/users/acc1c8b2d8c54b8b83614f7bf013ddb0 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:21:49.223023 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1403/2793] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:21:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:21:49.226015 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd496f74-af83-4a79-ae5e-a19e955c1910 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:49.226473 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd496f74-af83-4a79-ae5e-a19e955c1910 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:49.226751 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd496f74-af83-4a79-ae5e-a19e955c1910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:49.231292 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eeda72-5d23-4da1-b033-0442edf07b73 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:49.233002 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-31eeda72-5d23-4da1-b033-0442edf07b73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:49.246300 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-bd496f74-af83-4a79-ae5e-a19e955c1910 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:49.246509 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eeda72-5d23-4da1-b033-0442edf07b73 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:49.247548 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-bd496f74-af83-4a79-ae5e-a19e955c1910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:49.250054 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:49.277481 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:49.305778 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37287b69-2488-4e6a-954e-de0232bcbf4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:49.306382 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37287b69-2488-4e6a-954e-de0232bcbf4a admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:49.306657 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37287b69-2488-4e6a-954e-de0232bcbf4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:49.306956 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37287b69-2488-4e6a-954e-de0232bcbf4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:49.307582 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37287b69-2488-4e6a-954e-de0232bcbf4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:49.309122 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1391/2794] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:21:49.309489 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37287b69-2488-4e6a-954e-de0232bcbf4a admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:49.319224 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd496f74-af83-4a79-ae5e-a19e955c1910 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:49.321020 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-bd496f74-af83-4a79-ae5e-a19e955c1910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:49.340539 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1404/2795] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:21:49 2022] POST /identity/v3/projects => generated 365 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:21:49.341512 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37287b69-2488-4e6a-954e-de0232bcbf4a admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:49.381832 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd496f74-af83-4a79-ae5e-a19e955c1910 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:49.401422 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37287b69-2488-4e6a-954e-de0232bcbf4a admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:49.406659 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c2fdc-c258-419c-8bd5-eaa4a51ff5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:49.407284 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c2fdc-c258-419c-8bd5-eaa4a51ff5dd admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:49.407558 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c2fdc-c258-419c-8bd5-eaa4a51ff5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:49.407827 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c2fdc-c258-419c-8bd5-eaa4a51ff5dd admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:49.408449 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c2fdc-c258-419c-8bd5-eaa4a51ff5dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:49.410324 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c2fdc-c258-419c-8bd5-eaa4a51ff5dd admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:49.432047 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757fbb14-bc46-4431-9309-b71cbade47f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:49.432575 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fbb14-bc46-4431-9309-b71cbade47f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:49.432850 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fbb14-bc46-4431-9309-b71cbade47f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:49.433146 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fbb14-bc46-4431-9309-b71cbade47f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:49.436395 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1405/2796] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:21:49 2022] POST /identity/v3/users => generated 283 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:21:49.440368 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c2fdc-c258-419c-8bd5-eaa4a51ff5dd admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:49.443254 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757fbb14-bc46-4431-9309-b71cbade47f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:49.444723 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757fbb14-bc46-4431-9309-b71cbade47f3 service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:49.451358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c2fdc-c258-419c-8bd5-eaa4a51ff5dd admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:49.454703 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1392/2797] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:21:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:21:49.456184 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4296c06c-65e0-4c6b-9507-e12d70eefb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:49.456773 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4296c06c-65e0-4c6b-9507-e12d70eefb84 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:49.457053 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4296c06c-65e0-4c6b-9507-e12d70eefb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:49.457323 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4296c06c-65e0-4c6b-9507-e12d70eefb84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:49.457962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4296c06c-65e0-4c6b-9507-e12d70eefb84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:49.459586 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4296c06c-65e0-4c6b-9507-e12d70eefb84 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:49.471010 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1406/2798] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:21:49.474725 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757fbb14-bc46-4431-9309-b71cbade47f3 service neutron] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:49.484172 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757fbb14-bc46-4431-9309-b71cbade47f3 service neutron] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:49.488737 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273deec-3b3a-4e53-b7f6-4f07a89f8f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:49.489328 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273deec-3b3a-4e53-b7f6-4f07a89f8f20 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:49.489611 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273deec-3b3a-4e53-b7f6-4f07a89f8f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:49.489871 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273deec-3b3a-4e53-b7f6-4f07a89f8f20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:49.490503 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f273deec-3b3a-4e53-b7f6-4f07a89f8f20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:49.492122 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f273deec-3b3a-4e53-b7f6-4f07a89f8f20 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:49.502355 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1393/2799] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:21:49.506813 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4296c06c-65e0-4c6b-9507-e12d70eefb84 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:49.518414 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4296c06c-65e0-4c6b-9507-e12d70eefb84 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:49.523617 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da7c8a3-c512-42e3-809f-7af3c3a906b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:49.524544 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7c8a3-c512-42e3-809f-7af3c3a906b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:49.524913 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7c8a3-c512-42e3-809f-7af3c3a906b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:49.525192 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7c8a3-c512-42e3-809f-7af3c3a906b9 admin admin] PATH_INFO: `/v3/projects/35138dfc71394eb2981128de6d451acb/users/1134b8d4c16a4ba688c350402d984d08/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:49.541437 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da7c8a3-c512-42e3-809f-7af3c3a906b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35138dfc71394eb2981128de6d451acb, user_id=1134b8d4c16a4ba688c350402d984d08, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:49.544182 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da7c8a3-c512-42e3-809f-7af3c3a906b9 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:49.557390 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1407/2800] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:21:49 2022] PUT /identity/v3/projects/35138dfc71394eb2981128de6d451acb/users/1134b8d4c16a4ba688c350402d984d08/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:21:49.568244 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9ec1a-684e-4008-a52d-de3e4d869688 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:49.568526 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9ec1a-684e-4008-a52d-de3e4d869688 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:49.568799 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9ec1a-684e-4008-a52d-de3e4d869688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:49.599326 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-adb9ec1a-684e-4008-a52d-de3e4d869688 None None] MFA Rules not processed for user `1134b8d4c16a4ba688c350402d984d08`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:49.600573 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-adb9ec1a-684e-4008-a52d-de3e4d869688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:49.673384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1394/2801] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:49.677872 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da7c8a3-c512-42e3-809f-7af3c3a906b9 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:49.730791 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb9ec1a-684e-4008-a52d-de3e4d869688 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:49.732235 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-adb9ec1a-684e-4008-a52d-de3e4d869688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:49.734713 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:49.747285 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:49.755910 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da7c8a3-c512-42e3-809f-7af3c3a906b9 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:49.760811 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d069e60-b60a-41ce-98b2-47dfd4fe087e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:49.762056 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d069e60-b60a-41ce-98b2-47dfd4fe087e admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:49.762459 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d069e60-b60a-41ce-98b2-47dfd4fe087e admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:49.762712 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d069e60-b60a-41ce-98b2-47dfd4fe087e admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:49.763388 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d069e60-b60a-41ce-98b2-47dfd4fe087e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:49.765242 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d069e60-b60a-41ce-98b2-47dfd4fe087e admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:49.796037 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1408/2802] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:21:49 2022] POST /identity/v3/projects => generated 365 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:21:49.797034 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d069e60-b60a-41ce-98b2-47dfd4fe087e admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:49.802236 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb9ec1a-684e-4008-a52d-de3e4d869688 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:49.861513 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58144260-97b1-4066-b1a8-c397d9ae2ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:49.862141 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58144260-97b1-4066-b1a8-c397d9ae2ad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:49.862418 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58144260-97b1-4066-b1a8-c397d9ae2ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:49.862687 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58144260-97b1-4066-b1a8-c397d9ae2ad0 admin admin] PATH_INFO: `/v3/projects/01cbf4941c6c48b29a860150a980c2fa` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:49.862811 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d069e60-b60a-41ce-98b2-47dfd4fe087e admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:49.863837 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58144260-97b1-4066-b1a8-c397d9ae2ad0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01cbf4941c6c48b29a860150a980c2fa)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:49.865753 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58144260-97b1-4066-b1a8-c397d9ae2ad0 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:49.867431 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382cf234-33d5-43ca-8762-9773e16c85e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:49.868027 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382cf234-33d5-43ca-8762-9773e16c85e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:49.868302 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382cf234-33d5-43ca-8762-9773e16c85e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:49.868569 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382cf234-33d5-43ca-8762-9773e16c85e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:49.869239 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382cf234-33d5-43ca-8762-9773e16c85e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:49.871710 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382cf234-33d5-43ca-8762-9773e16c85e2 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:49.893085 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1409/2803] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:21:49 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:21:49.897884 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382cf234-33d5-43ca-8762-9773e16c85e2 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:49.908152 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382cf234-33d5-43ca-8762-9773e16c85e2 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:49.914282 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-58144260-97b1-4066-b1a8-c397d9ae2ad0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01cbf4941c6c48b29a860150a980c2fa', 'request_id': 'req-58144260-97b1-4066-b1a8-c397d9ae2ad0'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:49.919764 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-58144260-97b1-4066-b1a8-c397d9ae2ad0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01cbf4941c6c48b29a860150a980c2fa', 'request_id': 'req-58144260-97b1-4066-b1a8-c397d9ae2ad0'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:49.927943 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-58144260-97b1-4066-b1a8-c397d9ae2ad0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01cbf4941c6c48b29a860150a980c2fa', 'request_id': 'req-58144260-97b1-4066-b1a8-c397d9ae2ad0'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:49.939922 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-58144260-97b1-4066-b1a8-c397d9ae2ad0 admin admin] The token cache is being invalidate because project 01cbf4941c6c48b29a860150a980c2fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:21:49.940205 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-58144260-97b1-4066-b1a8-c397d9ae2ad0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:49.940850 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-58144260-97b1-4066-b1a8-c397d9ae2ad0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:49.942316 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1395/2804] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:21:49 2022] DELETE /identity/v3/projects/01cbf4941c6c48b29a860150a980c2fa => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:49.946232 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58144260-97b1-4066-b1a8-c397d9ae2ad0 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:49.947559 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-58144260-97b1-4066-b1a8-c397d9ae2ad0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:49.966630 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654fdac-0f59-4f66-b1f8-6fdbe5296346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:49.967274 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654fdac-0f59-4f66-b1f8-6fdbe5296346 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:49.967555 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654fdac-0f59-4f66-b1f8-6fdbe5296346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:49.967829 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654fdac-0f59-4f66-b1f8-6fdbe5296346 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:49.968480 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d654fdac-0f59-4f66-b1f8-6fdbe5296346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:49.970037 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d654fdac-0f59-4f66-b1f8-6fdbe5296346 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:49.979697 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1410/2805] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:21:49.983639 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654fdac-0f59-4f66-b1f8-6fdbe5296346 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:49.985277 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d654fdac-0f59-4f66-b1f8-6fdbe5296346 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:49.997811 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654fdac-0f59-4f66-b1f8-6fdbe5296346 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:49.997959 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:50.003520 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7606951-1d58-484c-abeb-2fcf26b31326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:50.004156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7606951-1d58-484c-abeb-2fcf26b31326 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:50.004437 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7606951-1d58-484c-abeb-2fcf26b31326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:50.004710 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7606951-1d58-484c-abeb-2fcf26b31326 admin admin] PATH_INFO: `/v3/projects/ff2b255c631c4e8f852db18ef7bace46/users/92be9bacc839486e9fc96eecb1bc3d6f/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:50.015306 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58144260-97b1-4066-b1a8-c397d9ae2ad0 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:50.020273 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7606951-1d58-484c-abeb-2fcf26b31326 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff2b255c631c4e8f852db18ef7bace46, user_id=92be9bacc839486e9fc96eecb1bc3d6f, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:50.020413 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c56ea2-4cb3-48b8-8c40-f138770001cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:50.020951 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c56ea2-4cb3-48b8-8c40-f138770001cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:50.021231 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c56ea2-4cb3-48b8-8c40-f138770001cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:50.021508 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c56ea2-4cb3-48b8-8c40-f138770001cf admin admin] PATH_INFO: `/v3/users/2348302e3df14e7f98186fea89a9af87` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:50.022644 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c56ea2-4cb3-48b8-8c40-f138770001cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=2348302e3df14e7f98186fea89a9af87)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:50.023180 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7606951-1d58-484c-abeb-2fcf26b31326 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:50.024561 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c56ea2-4cb3-48b8-8c40-f138770001cf admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:50.036581 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1411/2806] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:21:49 2022] PUT /identity/v3/projects/ff2b255c631c4e8f852db18ef7bace46/users/92be9bacc839486e9fc96eecb1bc3d6f/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:21:50.040483 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7606951-1d58-484c-abeb-2fcf26b31326 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:50.082447 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:50.083367 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-50c56ea2-4cb3-48b8-8c40-f138770001cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2348302e3df14e7f98186fea89a9af87'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:50.091732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-50c56ea2-4cb3-48b8-8c40-f138770001cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2348302e3df14e7f98186fea89a9af87'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:50.092831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-50c56ea2-4cb3-48b8-8c40-f138770001cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2348302e3df14e7f98186fea89a9af87'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:50.095495 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-50c56ea2-4cb3-48b8-8c40-f138770001cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2348302e3df14e7f98186fea89a9af87'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:50.102615 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-50c56ea2-4cb3-48b8-8c40-f138770001cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2348302e3df14e7f98186fea89a9af87'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:50.121272 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7606951-1d58-484c-abeb-2fcf26b31326 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:50.123442 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1396/2807] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:21:49 2022] DELETE /identity/v3/users/2348302e3df14e7f98186fea89a9af87 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:21:50.204739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32fe9c-a7a5-4c23-bc49-10a6746deaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:50.205474 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32fe9c-a7a5-4c23-bc49-10a6746deaf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:50.205756 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32fe9c-a7a5-4c23-bc49-10a6746deaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:50.206023 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32fe9c-a7a5-4c23-bc49-10a6746deaf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:50.206760 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f32fe9c-a7a5-4c23-bc49-10a6746deaf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:50.208564 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f32fe9c-a7a5-4c23-bc49-10a6746deaf3 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:50.221770 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1412/2808] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:50 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:21:50.223068 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c56ea2-4cb3-48b8-8c40-f138770001cf admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:50.224487 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-50c56ea2-4cb3-48b8-8c40-f138770001cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:50.239221 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c56ea2-4cb3-48b8-8c40-f138770001cf admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:50.246903 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b2d47-035d-4657-93bc-90928f127ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:50.247762 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b2d47-035d-4657-93bc-90928f127ad1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:50.248140 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b2d47-035d-4657-93bc-90928f127ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:50.248722 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b2d47-035d-4657-93bc-90928f127ad1 admin admin] PATH_INFO: `/v3/projects/ff2b255c631c4e8f852db18ef7bace46/users/92be9bacc839486e9fc96eecb1bc3d6f/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:50.251773 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b2d47-035d-4657-93bc-90928f127ad1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff2b255c631c4e8f852db18ef7bace46, user_id=92be9bacc839486e9fc96eecb1bc3d6f, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:50.254544 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b2d47-035d-4657-93bc-90928f127ad1 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:50.277620 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1397/2809] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:21:50 2022] PUT /identity/v3/projects/ff2b255c631c4e8f852db18ef7bace46/users/92be9bacc839486e9fc96eecb1bc3d6f/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:50.279449 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b8859-4f05-4670-9b42-f210067fcf6f None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:50.279806 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b8859-4f05-4670-9b42-f210067fcf6f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:50.280149 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b8859-4f05-4670-9b42-f210067fcf6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:50.320961 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-729b8859-4f05-4670-9b42-f210067fcf6f None None] MFA Rules not processed for user `92be9bacc839486e9fc96eecb1bc3d6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:50.322342 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-729b8859-4f05-4670-9b42-f210067fcf6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:50.402402 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1413/2810] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:50 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:50.414594 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b2d47-035d-4657-93bc-90928f127ad1 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:50.419230 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-bb1b2d47-035d-4657-93bc-90928f127ad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:50.444316 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b8859-4f05-4670-9b42-f210067fcf6f None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:50.445767 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-729b8859-4f05-4670-9b42-f210067fcf6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:50.500634 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b2d47-035d-4657-93bc-90928f127ad1 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:50.504578 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c1c0e9-88e0-4b38-8b75-a9a0c6d0913b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:50.505186 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1c0e9-88e0-4b38-8b75-a9a0c6d0913b service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:50.505582 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1c0e9-88e0-4b38-8b75-a9a0c6d0913b service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:50.505861 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1c0e9-88e0-4b38-8b75-a9a0c6d0913b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:50.518157 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c1c0e9-88e0-4b38-8b75-a9a0c6d0913b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:50.519729 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c1c0e9-88e0-4b38-8b75-a9a0c6d0913b service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:50.531070 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1398/2811] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:21:50 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:21:50.545584 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b8859-4f05-4670-9b42-f210067fcf6f None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:50.550760 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46ab8d-9680-4122-ae1c-1e3b092fc1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:50.551904 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46ab8d-9680-4122-ae1c-1e3b092fc1e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:50.551904 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46ab8d-9680-4122-ae1c-1e3b092fc1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:50.552136 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46ab8d-9680-4122-ae1c-1e3b092fc1e3 admin admin] PATH_INFO: `/v3/projects/0c207550dc874483aae3067151575d07` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:50.553450 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b46ab8d-9680-4122-ae1c-1e3b092fc1e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c207550dc874483aae3067151575d07)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:50.555676 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b46ab8d-9680-4122-ae1c-1e3b092fc1e3 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:50.559644 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:50.631400 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c1c0e9-88e0-4b38-8b75-a9a0c6d0913b service cinder] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:50.634188 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-4b46ab8d-9680-4122-ae1c-1e3b092fc1e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c207550dc874483aae3067151575d07', 'request_id': 'req-4b46ab8d-9680-4122-ae1c-1e3b092fc1e3'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:50.641803 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c1c0e9-88e0-4b38-8b75-a9a0c6d0913b service cinder] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:50.649907 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-4b46ab8d-9680-4122-ae1c-1e3b092fc1e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c207550dc874483aae3067151575d07', 'request_id': 'req-4b46ab8d-9680-4122-ae1c-1e3b092fc1e3'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:50.656875 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-4b46ab8d-9680-4122-ae1c-1e3b092fc1e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c207550dc874483aae3067151575d07', 'request_id': 'req-4b46ab8d-9680-4122-ae1c-1e3b092fc1e3'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:50.673180 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-4b46ab8d-9680-4122-ae1c-1e3b092fc1e3 admin admin] The token cache is being invalidate because project 0c207550dc874483aae3067151575d07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:21:50.673472 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-4b46ab8d-9680-4122-ae1c-1e3b092fc1e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:50.674149 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-4b46ab8d-9680-4122-ae1c-1e3b092fc1e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:50.675677 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1414/2812] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:21:50 2022] DELETE /identity/v3/projects/0c207550dc874483aae3067151575d07 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:21:50.678710 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46ab8d-9680-4122-ae1c-1e3b092fc1e3 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:50.678710 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-4b46ab8d-9680-4122-ae1c-1e3b092fc1e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:50.694930 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb053a-31ac-40d4-bf15-a7f644093458 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:50.695638 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb053a-31ac-40d4-bf15-a7f644093458 service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:50.695978 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb053a-31ac-40d4-bf15-a7f644093458 service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:50.696313 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb053a-31ac-40d4-bf15-a7f644093458 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:50.698174 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-edfb053a-31ac-40d4-bf15-a7f644093458 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:50.709476 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:50.745285 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46ab8d-9680-4122-ae1c-1e3b092fc1e3 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:50.750224 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff733d5a-ccc7-43ad-8e31-486ea0b3e539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:50.751012 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff733d5a-ccc7-43ad-8e31-486ea0b3e539 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:50.751279 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff733d5a-ccc7-43ad-8e31-486ea0b3e539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:50.751505 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff733d5a-ccc7-43ad-8e31-486ea0b3e539 admin admin] PATH_INFO: `/v3/users/4c26b531c07a498c9617c9ee2f7bf7e9` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:50.752647 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff733d5a-ccc7-43ad-8e31-486ea0b3e539 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c26b531c07a498c9617c9ee2f7bf7e9)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:50.754550 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff733d5a-ccc7-43ad-8e31-486ea0b3e539 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:50.761820 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb053a-31ac-40d4-bf15-a7f644093458 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:50.763320 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb053a-31ac-40d4-bf15-a7f644093458 service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:50.773029 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1399/2813] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:21:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:21:50.773914 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb053a-31ac-40d4-bf15-a7f644093458 service cinder] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:50.775301 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-edfb053a-31ac-40d4-bf15-a7f644093458 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:50.791475 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb053a-31ac-40d4-bf15-a7f644093458 service cinder] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:50.797267 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f8dad-f5de-44b7-992c-8b8c53a5ee1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:50.797979 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f8dad-f5de-44b7-992c-8b8c53a5ee1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:50.798341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f8dad-f5de-44b7-992c-8b8c53a5ee1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:50.798685 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f8dad-f5de-44b7-992c-8b8c53a5ee1f admin admin] PATH_INFO: `/v3/users/5f717ee157f04aa78d0dd37c978b8f78` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:50.800002 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87f8dad-f5de-44b7-992c-8b8c53a5ee1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f717ee157f04aa78d0dd37c978b8f78)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:50.801982 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87f8dad-f5de-44b7-992c-8b8c53a5ee1f admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:50.808952 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-ff733d5a-ccc7-43ad-8e31-486ea0b3e539 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c26b531c07a498c9617c9ee2f7bf7e9'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:50.818076 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-ff733d5a-ccc7-43ad-8e31-486ea0b3e539 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c26b531c07a498c9617c9ee2f7bf7e9'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:50.819014 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-ff733d5a-ccc7-43ad-8e31-486ea0b3e539 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c26b531c07a498c9617c9ee2f7bf7e9'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:50.819695 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-ff733d5a-ccc7-43ad-8e31-486ea0b3e539 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c26b531c07a498c9617c9ee2f7bf7e9'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:50.830940 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-ff733d5a-ccc7-43ad-8e31-486ea0b3e539 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c26b531c07a498c9617c9ee2f7bf7e9'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:50.857169 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1415/2814] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:21:50 2022] DELETE /identity/v3/users/4c26b531c07a498c9617c9ee2f7bf7e9 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:50.864245 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-c87f8dad-f5de-44b7-992c-8b8c53a5ee1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f717ee157f04aa78d0dd37c978b8f78'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:50.873033 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-c87f8dad-f5de-44b7-992c-8b8c53a5ee1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f717ee157f04aa78d0dd37c978b8f78'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:50.874027 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-c87f8dad-f5de-44b7-992c-8b8c53a5ee1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f717ee157f04aa78d0dd37c978b8f78'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:50.874730 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-c87f8dad-f5de-44b7-992c-8b8c53a5ee1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f717ee157f04aa78d0dd37c978b8f78'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:50.883524 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-c87f8dad-f5de-44b7-992c-8b8c53a5ee1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f717ee157f04aa78d0dd37c978b8f78'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:50.905414 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1400/2815] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:21:50 2022] DELETE /identity/v3/users/5f717ee157f04aa78d0dd37c978b8f78 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:51.089150 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff733d5a-ccc7-43ad-8e31-486ea0b3e539 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:51.091418 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ff733d5a-ccc7-43ad-8e31-486ea0b3e539 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:51.127408 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:51.199511 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff733d5a-ccc7-43ad-8e31-486ea0b3e539 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:51.247544 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f8dad-f5de-44b7-992c-8b8c53a5ee1f admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:51.248978 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-c87f8dad-f5de-44b7-992c-8b8c53a5ee1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:51.260290 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eb0140-9ff7-4692-9a13-7df7c8af024b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:51.261132 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb0140-9ff7-4692-9a13-7df7c8af024b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:51.261838 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb0140-9ff7-4692-9a13-7df7c8af024b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:51.262125 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb0140-9ff7-4692-9a13-7df7c8af024b admin admin] PATH_INFO: `/v3/projects/7a92484ba35b46b6bd86a3072403410e` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:51.263371 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eb0140-9ff7-4692-9a13-7df7c8af024b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a92484ba35b46b6bd86a3072403410e)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:51.265259 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eb0140-9ff7-4692-9a13-7df7c8af024b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:51.315211 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-92eb0140-9ff7-4692-9a13-7df7c8af024b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a92484ba35b46b6bd86a3072403410e', 'request_id': 'req-92eb0140-9ff7-4692-9a13-7df7c8af024b'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:51.322991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-92eb0140-9ff7-4692-9a13-7df7c8af024b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a92484ba35b46b6bd86a3072403410e', 'request_id': 'req-92eb0140-9ff7-4692-9a13-7df7c8af024b'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:51.329071 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f8dad-f5de-44b7-992c-8b8c53a5ee1f admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:51.329570 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-92eb0140-9ff7-4692-9a13-7df7c8af024b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a92484ba35b46b6bd86a3072403410e', 'request_id': 'req-92eb0140-9ff7-4692-9a13-7df7c8af024b'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:51.341086 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-92eb0140-9ff7-4692-9a13-7df7c8af024b admin admin] The token cache is being invalidate because project 7a92484ba35b46b6bd86a3072403410e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:21:51.341384 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-92eb0140-9ff7-4692-9a13-7df7c8af024b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:51.342065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-92eb0140-9ff7-4692-9a13-7df7c8af024b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:51.343628 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1416/2816] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:21:51 2022] DELETE /identity/v3/projects/7a92484ba35b46b6bd86a3072403410e => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:51.348965 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f837a9-1b29-4580-88b0-09d8dff8d960 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:51.349378 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f837a9-1b29-4580-88b0-09d8dff8d960 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:51.349651 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f837a9-1b29-4580-88b0-09d8dff8d960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:51.357894 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:51.368884 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-e1f837a9-1b29-4580-88b0-09d8dff8d960 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:51.370105 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e1f837a9-1b29-4580-88b0-09d8dff8d960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:51.390439 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266b5929-3a1b-48ef-ad64-ff50a23f0dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:51.391366 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b5929-3a1b-48ef-ad64-ff50a23f0dac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:51.391848 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b5929-3a1b-48ef-ad64-ff50a23f0dac admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:51.392217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b5929-3a1b-48ef-ad64-ff50a23f0dac admin admin] PATH_INFO: `/v3/projects/dae1fabb3bd2430494d688094730994a` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:51.393416 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266b5929-3a1b-48ef-ad64-ff50a23f0dac admin admin] RBAC: Authorizing `identity:delete_project(project_id=dae1fabb3bd2430494d688094730994a)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:51.395795 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266b5929-3a1b-48ef-ad64-ff50a23f0dac admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:51.415776 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:51.436228 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1417/2817] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:21:51.446077 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b3866-c20a-4b5c-8939-fa5e3ad033e9 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:51.446375 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b3866-c20a-4b5c-8939-fa5e3ad033e9 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:51.446690 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b3866-c20a-4b5c-8939-fa5e3ad033e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:51.448027 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-266b5929-3a1b-48ef-ad64-ff50a23f0dac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dae1fabb3bd2430494d688094730994a', 'request_id': 'req-266b5929-3a1b-48ef-ad64-ff50a23f0dac'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:51.454703 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-266b5929-3a1b-48ef-ad64-ff50a23f0dac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dae1fabb3bd2430494d688094730994a', 'request_id': 'req-266b5929-3a1b-48ef-ad64-ff50a23f0dac'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:51.462750 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-266b5929-3a1b-48ef-ad64-ff50a23f0dac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dae1fabb3bd2430494d688094730994a', 'request_id': 'req-266b5929-3a1b-48ef-ad64-ff50a23f0dac'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:51.467030 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-574b3866-c20a-4b5c-8939-fa5e3ad033e9 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:51.468237 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-574b3866-c20a-4b5c-8939-fa5e3ad033e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:51.475432 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-266b5929-3a1b-48ef-ad64-ff50a23f0dac admin admin] The token cache is being invalidate because project dae1fabb3bd2430494d688094730994a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:21:51.475720 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-266b5929-3a1b-48ef-ad64-ff50a23f0dac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:51.476345 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-266b5929-3a1b-48ef-ad64-ff50a23f0dac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:51.477823 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1401/2818] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:21:51 2022] DELETE /identity/v3/projects/dae1fabb3bd2430494d688094730994a => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:51.482325 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a3d7b-bf1d-4e82-bdb3-92cdd6e071b2 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:51.482602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a3d7b-bf1d-4e82-bdb3-92cdd6e071b2 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:51.482900 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a3d7b-bf1d-4e82-bdb3-92cdd6e071b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:51.502072 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-3f5a3d7b-bf1d-4e82-bdb3-92cdd6e071b2 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:51.503291 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-3f5a3d7b-bf1d-4e82-bdb3-92cdd6e071b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:51.531217 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1418/2819] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:51.535461 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b3866-c20a-4b5c-8939-fa5e3ad033e9 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:51.548305 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b3866-c20a-4b5c-8939-fa5e3ad033e9 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:51.552934 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89672395-4927-4aeb-92f5-bea32cc61eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:51.553525 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89672395-4927-4aeb-92f5-bea32cc61eca admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:51.553803 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89672395-4927-4aeb-92f5-bea32cc61eca admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:51.554072 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89672395-4927-4aeb-92f5-bea32cc61eca admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:51.554714 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89672395-4927-4aeb-92f5-bea32cc61eca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:21:51.555053 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89672395-4927-4aeb-92f5-bea32cc61eca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:51.556625 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89672395-4927-4aeb-92f5-bea32cc61eca admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:51.566291 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1402/2820] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:21:51.574584 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1419/2821] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:21:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:21:51.579019 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a3d7b-bf1d-4e82-bdb3-92cdd6e071b2 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:51.589776 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a3d7b-bf1d-4e82-bdb3-92cdd6e071b2 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:51.594952 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb430184-34a9-4cd0-8e59-cc1b204f5e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:51.595589 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb430184-34a9-4cd0-8e59-cc1b204f5e8f admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:51.595866 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb430184-34a9-4cd0-8e59-cc1b204f5e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:51.596137 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb430184-34a9-4cd0-8e59-cc1b204f5e8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:51.596828 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb430184-34a9-4cd0-8e59-cc1b204f5e8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:51.598794 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb430184-34a9-4cd0-8e59-cc1b204f5e8f admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:51.631666 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1403/2822] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:21:51 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:21:51.633109 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89672395-4927-4aeb-92f5-bea32cc61eca admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:51.634141 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb9f6b-bb18-4c10-b39a-0341ead3a33c None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:51.634458 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb9f6b-bb18-4c10-b39a-0341ead3a33c None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:51.634899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb9f6b-bb18-4c10-b39a-0341ead3a33c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:51.644653 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89672395-4927-4aeb-92f5-bea32cc61eca admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:51.656107 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-6eeb9f6b-bb18-4c10-b39a-0341ead3a33c None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:51.657307 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-6eeb9f6b-bb18-4c10-b39a-0341ead3a33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:51.692578 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:51.705230 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0228cf-cdba-4108-9336-483f3fedd2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:51.705849 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0228cf-cdba-4108-9336-483f3fedd2ef admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:51.706126 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0228cf-cdba-4108-9336-483f3fedd2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:51.706397 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0228cf-cdba-4108-9336-483f3fedd2ef admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:51.707065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0228cf-cdba-4108-9336-483f3fedd2ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:51.708989 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0228cf-cdba-4108-9336-483f3fedd2ef admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:51.719991 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1404/2823] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:21:51.724958 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeb9f6b-bb18-4c10-b39a-0341ead3a33c None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:51.734559 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1420/2824] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:21:51 2022] POST /identity/v3/users => generated 283 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:21:51.735612 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0228cf-cdba-4108-9336-483f3fedd2ef admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:51.736113 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeb9f6b-bb18-4c10-b39a-0341ead3a33c None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:51.740761 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644ae63-3cd1-48f7-8131-4b395922cb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:51.741355 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644ae63-3cd1-48f7-8131-4b395922cb90 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:51.741996 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644ae63-3cd1-48f7-8131-4b395922cb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:51.741996 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644ae63-3cd1-48f7-8131-4b395922cb90 admin admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:51.742548 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2644ae63-3cd1-48f7-8131-4b395922cb90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:21:51.742859 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2644ae63-3cd1-48f7-8131-4b395922cb90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:51.744477 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2644ae63-3cd1-48f7-8131-4b395922cb90 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:51.746131 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0228cf-cdba-4108-9336-483f3fedd2ef admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:51.751027 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757f7e03-e13d-4397-a057-7920b8c091d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:51.751624 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f7e03-e13d-4397-a057-7920b8c091d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:51.751902 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f7e03-e13d-4397-a057-7920b8c091d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:51.752175 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f7e03-e13d-4397-a057-7920b8c091d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:51.753300 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757f7e03-e13d-4397-a057-7920b8c091d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:51.754978 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757f7e03-e13d-4397-a057-7920b8c091d2 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:51.757455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1405/2825] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:21:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:21:51.761577 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644ae63-3cd1-48f7-8131-4b395922cb90 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:51.767949 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1421/2826] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:21:51.771282 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644ae63-3cd1-48f7-8131-4b395922cb90 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:51.771721 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757f7e03-e13d-4397-a057-7920b8c091d2 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:51.775863 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc51a65-b07e-4e7f-884f-46e4dac28583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:51.776622 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc51a65-b07e-4e7f-884f-46e4dac28583 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:51.776900 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc51a65-b07e-4e7f-884f-46e4dac28583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:51.777174 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc51a65-b07e-4e7f-884f-46e4dac28583 admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:51.778354 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc51a65-b07e-4e7f-884f-46e4dac28583 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:51.780299 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc51a65-b07e-4e7f-884f-46e4dac28583 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:51.781155 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757f7e03-e13d-4397-a057-7920b8c091d2 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:51.785751 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5099c5f-fc62-4e83-9413-bd1eeb70c3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:51.786314 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5099c5f-fc62-4e83-9413-bd1eeb70c3ad admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:51.786603 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5099c5f-fc62-4e83-9413-bd1eeb70c3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:51.786900 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5099c5f-fc62-4e83-9413-bd1eeb70c3ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:51.787528 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5099c5f-fc62-4e83-9413-bd1eeb70c3ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:51.789060 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5099c5f-fc62-4e83-9413-bd1eeb70c3ad admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:51.799379 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1422/2827] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:21:51.804021 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5099c5f-fc62-4e83-9413-bd1eeb70c3ad admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:51.810442 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1406/2828] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:21:51 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:21:51.814857 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc51a65-b07e-4e7f-884f-46e4dac28583 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:51.817372 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5099c5f-fc62-4e83-9413-bd1eeb70c3ad admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:51.865347 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:51.882142 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d625be0-58cf-4529-b870-fc56012c4aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:51.882902 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d625be0-58cf-4529-b870-fc56012c4aee admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:51.883184 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d625be0-58cf-4529-b870-fc56012c4aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:51.883454 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d625be0-58cf-4529-b870-fc56012c4aee admin admin] PATH_INFO: `/v3/projects/efc2b68289774509abef904add3762d1/users/63ef31be1a4f4bf5b6a02acd42d68ee8/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:51.884658 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc51a65-b07e-4e7f-884f-46e4dac28583 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:51.889369 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689bf42c-5926-490c-a706-8ea75ecaf774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:51.889965 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689bf42c-5926-490c-a706-8ea75ecaf774 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:51.890245 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689bf42c-5926-490c-a706-8ea75ecaf774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:51.890517 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689bf42c-5926-490c-a706-8ea75ecaf774 admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:51.891172 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689bf42c-5926-490c-a706-8ea75ecaf774 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:51.893083 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689bf42c-5926-490c-a706-8ea75ecaf774 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:51.898010 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d625be0-58cf-4529-b870-fc56012c4aee admin admin] RBAC: Authorizing `identity:create_grant(project_id=efc2b68289774509abef904add3762d1, user_id=63ef31be1a4f4bf5b6a02acd42d68ee8, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:51.900652 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d625be0-58cf-4529-b870-fc56012c4aee admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:51.913403 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1423/2829] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:21:51 2022] PUT /identity/v3/projects/efc2b68289774509abef904add3762d1/users/63ef31be1a4f4bf5b6a02acd42d68ee8/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:21:51.923761 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1407/2830] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:21:51 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:21:51.923879 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d625be0-58cf-4529-b870-fc56012c4aee admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:51.925755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc460d-c4a7-4b10-bfa4-eda4ce1729f2 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:51.926041 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc460d-c4a7-4b10-bfa4-eda4ce1729f2 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:51.926315 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc460d-c4a7-4b10-bfa4-eda4ce1729f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:51.954686 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-46fc460d-c4a7-4b10-bfa4-eda4ce1729f2 None None] MFA Rules not processed for user `63ef31be1a4f4bf5b6a02acd42d68ee8`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:51.955929 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-46fc460d-c4a7-4b10-bfa4-eda4ce1729f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:51.980944 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d625be0-58cf-4529-b870-fc56012c4aee admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:51.985672 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1168be0-cebf-4361-9f3a-e38bc3019dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:51.986276 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1168be0-cebf-4361-9f3a-e38bc3019dd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:51.986559 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1168be0-cebf-4361-9f3a-e38bc3019dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:51.986832 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1168be0-cebf-4361-9f3a-e38bc3019dd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:51.987497 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1168be0-cebf-4361-9f3a-e38bc3019dd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:51.989080 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1168be0-cebf-4361-9f3a-e38bc3019dd9 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:51.999429 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1424/2831] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:21:52.003493 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1168be0-cebf-4361-9f3a-e38bc3019dd9 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:52.012960 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1168be0-cebf-4361-9f3a-e38bc3019dd9 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:52.017567 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203c85e-afa1-4692-9497-93f6c6e98005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:52.018141 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203c85e-afa1-4692-9497-93f6c6e98005 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:52.018422 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203c85e-afa1-4692-9497-93f6c6e98005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:52.018695 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203c85e-afa1-4692-9497-93f6c6e98005 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:52.019349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a203c85e-afa1-4692-9497-93f6c6e98005 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:52.020226 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1408/2832] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:21:52.020899 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a203c85e-afa1-4692-9497-93f6c6e98005 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:52.034250 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1425/2833] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:52 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:21:52.034423 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fc460d-c4a7-4b10-bfa4-eda4ce1729f2 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:52.034423 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-46fc460d-c4a7-4b10-bfa4-eda4ce1729f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:52.035242 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203c85e-afa1-4692-9497-93f6c6e98005 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:52.045207 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203c85e-afa1-4692-9497-93f6c6e98005 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:52.049947 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08affa90-ec26-4cb7-872a-a662ca3dede9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:52.050565 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08affa90-ec26-4cb7-872a-a662ca3dede9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:52.050873 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08affa90-ec26-4cb7-872a-a662ca3dede9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:52.051156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08affa90-ec26-4cb7-872a-a662ca3dede9 admin admin] PATH_INFO: `/v3/projects/1bc077285f3a4fda895f9e9d6e3886fb/users/d846e1dac9c844c9add7e73da3b1e145/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:52.062026 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:52.066845 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08affa90-ec26-4cb7-872a-a662ca3dede9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bc077285f3a4fda895f9e9d6e3886fb, user_id=d846e1dac9c844c9add7e73da3b1e145, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:52.069501 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08affa90-ec26-4cb7-872a-a662ca3dede9 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:52.083434 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1426/2834] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:21:52 2022] PUT /identity/v3/projects/1bc077285f3a4fda895f9e9d6e3886fb/users/d846e1dac9c844c9add7e73da3b1e145/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:21:52.095529 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82394fc3-b23d-4224-a756-e5145b546980 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:52.095814 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82394fc3-b23d-4224-a756-e5145b546980 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:52.096704 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82394fc3-b23d-4224-a756-e5145b546980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:52.100657 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fc460d-c4a7-4b10-bfa4-eda4ce1729f2 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:52.125675 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-82394fc3-b23d-4224-a756-e5145b546980 None None] MFA Rules not processed for user `d846e1dac9c844c9add7e73da3b1e145`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:52.127104 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-82394fc3-b23d-4224-a756-e5145b546980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:52.153117 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd92be1-eed7-4820-91a3-d085a97fda09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:52.153794 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd92be1-eed7-4820-91a3-d085a97fda09 service cinder] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:52.154077 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd92be1-eed7-4820-91a3-d085a97fda09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:52.154350 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd92be1-eed7-4820-91a3-d085a97fda09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:52.206531 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1427/2835] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:52.223724 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd92be1-eed7-4820-91a3-d085a97fda09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:52.225192 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd92be1-eed7-4820-91a3-d085a97fda09 service cinder] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:52.234711 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1409/2836] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:21:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:21:52.379547 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82394fc3-b23d-4224-a756-e5145b546980 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:52.380979 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-82394fc3-b23d-4224-a756-e5145b546980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:52.397962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82394fc3-b23d-4224-a756-e5145b546980 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:52.403609 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174b3aa3-7e27-4183-afad-963b04e5aff8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:52.404281 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b3aa3-7e27-4183-afad-963b04e5aff8 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:52.404569 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b3aa3-7e27-4183-afad-963b04e5aff8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:52.404913 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b3aa3-7e27-4183-afad-963b04e5aff8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:52.414687 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b3aa3-7e27-4183-afad-963b04e5aff8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:52.416166 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b3aa3-7e27-4183-afad-963b04e5aff8 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:52.427550 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1428/2837] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:21:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:21:54.717992 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd92be1-eed7-4820-91a3-d085a97fda09 service cinder] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:54.719552 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-8fd92be1-eed7-4820-91a3-d085a97fda09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:54.783449 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd92be1-eed7-4820-91a3-d085a97fda09 service cinder] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:54.787897 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a83ff3-c4d9-4f9b-9037-cf1e01aaa2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:54.788530 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a83ff3-c4d9-4f9b-9037-cf1e01aaa2aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:54.788826 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a83ff3-c4d9-4f9b-9037-cf1e01aaa2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:54.789101 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a83ff3-c4d9-4f9b-9037-cf1e01aaa2aa admin admin] PATH_INFO: `/v3/users/1134b8d4c16a4ba688c350402d984d08` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:54.790288 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a83ff3-c4d9-4f9b-9037-cf1e01aaa2aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=1134b8d4c16a4ba688c350402d984d08)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:54.792230 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a83ff3-c4d9-4f9b-9037-cf1e01aaa2aa admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:54.831436 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:54.841521 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-f8a83ff3-c4d9-4f9b-9037-cf1e01aaa2aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1134b8d4c16a4ba688c350402d984d08'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:54.848157 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-f8a83ff3-c4d9-4f9b-9037-cf1e01aaa2aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1134b8d4c16a4ba688c350402d984d08'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:54.848791 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-f8a83ff3-c4d9-4f9b-9037-cf1e01aaa2aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1134b8d4c16a4ba688c350402d984d08'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:54.849392 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-f8a83ff3-c4d9-4f9b-9037-cf1e01aaa2aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1134b8d4c16a4ba688c350402d984d08'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:54.856239 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-f8a83ff3-c4d9-4f9b-9037-cf1e01aaa2aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1134b8d4c16a4ba688c350402d984d08'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:54.872977 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1410/2838] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:21:54 2022] DELETE /identity/v3/users/1134b8d4c16a4ba688c350402d984d08 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:54.877557 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86326e9-244c-4be1-9a8e-38f56a96d470 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:54.878033 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86326e9-244c-4be1-9a8e-38f56a96d470 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:54.878141 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86326e9-244c-4be1-9a8e-38f56a96d470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:54.885867 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:54.898093 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-a86326e9-244c-4be1-9a8e-38f56a96d470 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:54.899308 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-a86326e9-244c-4be1-9a8e-38f56a96d470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:54.957403 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1429/2839] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:21:54.967669 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a83ff3-c4d9-4f9b-9037-cf1e01aaa2aa admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:54.969067 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-f8a83ff3-c4d9-4f9b-9037-cf1e01aaa2aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:55.025920 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a83ff3-c4d9-4f9b-9037-cf1e01aaa2aa admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:55.029539 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47719371-0355-4041-b133-7e30a68b6069 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:55.030131 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47719371-0355-4041-b133-7e30a68b6069 service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:55.030408 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47719371-0355-4041-b133-7e30a68b6069 service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:55.030678 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47719371-0355-4041-b133-7e30a68b6069 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:55.039687 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47719371-0355-4041-b133-7e30a68b6069 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:55.041118 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47719371-0355-4041-b133-7e30a68b6069 service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:55.050705 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1411/2840] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:21:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:21:55.068682 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86326e9-244c-4be1-9a8e-38f56a96d470 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:55.070072 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-a86326e9-244c-4be1-9a8e-38f56a96d470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:55.082747 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86326e9-244c-4be1-9a8e-38f56a96d470 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:55.087366 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb632a4-7ad6-4cf1-a4cc-5bdb38c3bc29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:55.087988 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb632a4-7ad6-4cf1-a4cc-5bdb38c3bc29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:55.088264 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb632a4-7ad6-4cf1-a4cc-5bdb38c3bc29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:55.088535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb632a4-7ad6-4cf1-a4cc-5bdb38c3bc29 admin admin] PATH_INFO: `/v3/users/d846e1dac9c844c9add7e73da3b1e145` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:55.089661 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb632a4-7ad6-4cf1-a4cc-5bdb38c3bc29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d846e1dac9c844c9add7e73da3b1e145)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:55.091587 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb632a4-7ad6-4cf1-a4cc-5bdb38c3bc29 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:55.139285 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-deb632a4-7ad6-4cf1-a4cc-5bdb38c3bc29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd846e1dac9c844c9add7e73da3b1e145'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:55.147764 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-deb632a4-7ad6-4cf1-a4cc-5bdb38c3bc29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd846e1dac9c844c9add7e73da3b1e145'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:55.148512 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-deb632a4-7ad6-4cf1-a4cc-5bdb38c3bc29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd846e1dac9c844c9add7e73da3b1e145'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:55.149136 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-deb632a4-7ad6-4cf1-a4cc-5bdb38c3bc29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd846e1dac9c844c9add7e73da3b1e145'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:55.156927 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-deb632a4-7ad6-4cf1-a4cc-5bdb38c3bc29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd846e1dac9c844c9add7e73da3b1e145'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:55.176245 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1430/2841] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:21:55 2022] DELETE /identity/v3/users/d846e1dac9c844c9add7e73da3b1e145 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:55.180740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54359fa-b9c4-4864-b6da-ce573e215559 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:55.181079 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54359fa-b9c4-4864-b6da-ce573e215559 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:55.181353 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54359fa-b9c4-4864-b6da-ce573e215559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:55.223091 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-f54359fa-b9c4-4864-b6da-ce573e215559 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:55.224381 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-f54359fa-b9c4-4864-b6da-ce573e215559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:55.295846 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1412/2842] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:21:55.305315 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb632a4-7ad6-4cf1-a4cc-5bdb38c3bc29 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:55.307515 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-deb632a4-7ad6-4cf1-a4cc-5bdb38c3bc29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:55.335947 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54359fa-b9c4-4864-b6da-ce573e215559 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:55.337697 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-f54359fa-b9c4-4864-b6da-ce573e215559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:55.350259 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54359fa-b9c4-4864-b6da-ce573e215559 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:55.360615 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7f6f2e-990c-44f0-b075-5154d9a78d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:55.361256 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f6f2e-990c-44f0-b075-5154d9a78d2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:55.361531 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f6f2e-990c-44f0-b075-5154d9a78d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:55.361798 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f6f2e-990c-44f0-b075-5154d9a78d2c admin admin] PATH_INFO: `/v3/projects/35138dfc71394eb2981128de6d451acb` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:55.363049 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7f6f2e-990c-44f0-b075-5154d9a78d2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=35138dfc71394eb2981128de6d451acb)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:55.364961 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7f6f2e-990c-44f0-b075-5154d9a78d2c admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:55.389142 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb632a4-7ad6-4cf1-a4cc-5bdb38c3bc29 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:55.393033 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616eefe-998c-4212-b1be-2ab8b4124d52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:55.393641 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616eefe-998c-4212-b1be-2ab8b4124d52 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:55.393922 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616eefe-998c-4212-b1be-2ab8b4124d52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:55.394190 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616eefe-998c-4212-b1be-2ab8b4124d52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:55.407574 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d616eefe-998c-4212-b1be-2ab8b4124d52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:55.408996 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d616eefe-998c-4212-b1be-2ab8b4124d52 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:55.424359 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1431/2843] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:21:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:21:55.424542 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-fe7f6f2e-990c-44f0-b075-5154d9a78d2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35138dfc71394eb2981128de6d451acb', 'request_id': 'req-fe7f6f2e-990c-44f0-b075-5154d9a78d2c'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:55.427190 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:55.431410 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-fe7f6f2e-990c-44f0-b075-5154d9a78d2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35138dfc71394eb2981128de6d451acb', 'request_id': 'req-fe7f6f2e-990c-44f0-b075-5154d9a78d2c'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:55.441518 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-fe7f6f2e-990c-44f0-b075-5154d9a78d2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35138dfc71394eb2981128de6d451acb', 'request_id': 'req-fe7f6f2e-990c-44f0-b075-5154d9a78d2c'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:55.457011 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-fe7f6f2e-990c-44f0-b075-5154d9a78d2c admin admin] The token cache is being invalidate because project 35138dfc71394eb2981128de6d451acb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:21:55.457526 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-fe7f6f2e-990c-44f0-b075-5154d9a78d2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:55.458416 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-fe7f6f2e-990c-44f0-b075-5154d9a78d2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:55.460105 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1413/2844] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:21:55 2022] DELETE /identity/v3/projects/35138dfc71394eb2981128de6d451acb => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:21:55.464166 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616eefe-998c-4212-b1be-2ab8b4124d52 service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:55.474275 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616eefe-998c-4212-b1be-2ab8b4124d52 service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:55.492591 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:55.545405 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa48ed-8c3e-4ba2-896f-29a13b010106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:55.546352 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa48ed-8c3e-4ba2-896f-29a13b010106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:55.546710 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa48ed-8c3e-4ba2-896f-29a13b010106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:55.547092 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa48ed-8c3e-4ba2-896f-29a13b010106 admin admin] PATH_INFO: `/v3/users/92be9bacc839486e9fc96eecb1bc3d6f` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:55.548315 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aa48ed-8c3e-4ba2-896f-29a13b010106 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92be9bacc839486e9fc96eecb1bc3d6f)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:55.550291 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aa48ed-8c3e-4ba2-896f-29a13b010106 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:55.609446 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e6aa48ed-8c3e-4ba2-896f-29a13b010106 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92be9bacc839486e9fc96eecb1bc3d6f'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:55.617923 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e6aa48ed-8c3e-4ba2-896f-29a13b010106 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92be9bacc839486e9fc96eecb1bc3d6f'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:55.618678 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e6aa48ed-8c3e-4ba2-896f-29a13b010106 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92be9bacc839486e9fc96eecb1bc3d6f'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:55.619361 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e6aa48ed-8c3e-4ba2-896f-29a13b010106 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92be9bacc839486e9fc96eecb1bc3d6f'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:55.629384 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e6aa48ed-8c3e-4ba2-896f-29a13b010106 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92be9bacc839486e9fc96eecb1bc3d6f'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:55.652120 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1432/2845] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:21:55 2022] DELETE /identity/v3/users/92be9bacc839486e9fc96eecb1bc3d6f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:55.734750 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7f6f2e-990c-44f0-b075-5154d9a78d2c admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:55.736999 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-fe7f6f2e-990c-44f0-b075-5154d9a78d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:55.757493 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7f6f2e-990c-44f0-b075-5154d9a78d2c admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:55.833391 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5f48a-bc75-40bf-b01e-876790868db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:55.834046 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5f48a-bc75-40bf-b01e-876790868db7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:55.834329 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5f48a-bc75-40bf-b01e-876790868db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:55.834732 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5f48a-bc75-40bf-b01e-876790868db7 admin admin] PATH_INFO: `/v3/projects/1bc077285f3a4fda895f9e9d6e3886fb` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:55.835908 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c5f48a-bc75-40bf-b01e-876790868db7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bc077285f3a4fda895f9e9d6e3886fb)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:55.837822 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c5f48a-bc75-40bf-b01e-876790868db7 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:55.884998 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-a0c5f48a-bc75-40bf-b01e-876790868db7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bc077285f3a4fda895f9e9d6e3886fb', 'request_id': 'req-a0c5f48a-bc75-40bf-b01e-876790868db7'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:55.890422 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-a0c5f48a-bc75-40bf-b01e-876790868db7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bc077285f3a4fda895f9e9d6e3886fb', 'request_id': 'req-a0c5f48a-bc75-40bf-b01e-876790868db7'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:55.898213 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-a0c5f48a-bc75-40bf-b01e-876790868db7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bc077285f3a4fda895f9e9d6e3886fb', 'request_id': 'req-a0c5f48a-bc75-40bf-b01e-876790868db7'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:55.910715 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-a0c5f48a-bc75-40bf-b01e-876790868db7 admin admin] The token cache is being invalidate because project 1bc077285f3a4fda895f9e9d6e3886fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:21:55.911044 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-a0c5f48a-bc75-40bf-b01e-876790868db7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:55.911695 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-a0c5f48a-bc75-40bf-b01e-876790868db7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:55.913436 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1414/2846] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:21:55 2022] DELETE /identity/v3/projects/1bc077285f3a4fda895f9e9d6e3886fb => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:55.919096 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee582a-e31d-4f58-b721-d460bc4918a4 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:55.919611 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee582a-e31d-4f58-b721-d460bc4918a4 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:55.919812 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee582a-e31d-4f58-b721-d460bc4918a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:55.939624 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5f48a-bc75-40bf-b01e-876790868db7 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:55.941479 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-a0c5f48a-bc75-40bf-b01e-876790868db7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:55.945070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-20ee582a-e31d-4f58-b721-d460bc4918a4 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:55.946270 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-20ee582a-e31d-4f58-b721-d460bc4918a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:56.011156 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5f48a-bc75-40bf-b01e-876790868db7 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:56.012163 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1433/2847] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:56.016032 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77a03e-0b12-4b45-9a29-c8adeb5033ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:56.016673 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77a03e-0b12-4b45-9a29-c8adeb5033ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:56.016952 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77a03e-0b12-4b45-9a29-c8adeb5033ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:56.017265 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77a03e-0b12-4b45-9a29-c8adeb5033ca admin admin] PATH_INFO: `/v3/projects/ff2b255c631c4e8f852db18ef7bace46` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:56.018377 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce77a03e-0b12-4b45-9a29-c8adeb5033ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff2b255c631c4e8f852db18ef7bace46)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:56.019850 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb466a-86c4-4917-88ac-3ec88483aa1c None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:56.020137 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb466a-86c4-4917-88ac-3ec88483aa1c None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:56.020258 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce77a03e-0b12-4b45-9a29-c8adeb5033ca admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:56.020411 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb466a-86c4-4917-88ac-3ec88483aa1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:56.032443 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:56.040941 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-d0eb466a-86c4-4917-88ac-3ec88483aa1c None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:56.042198 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d0eb466a-86c4-4917-88ac-3ec88483aa1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:56.051535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1434/2848] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:21:56.055780 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb466a-86c4-4917-88ac-3ec88483aa1c None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:56.065132 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb466a-86c4-4917-88ac-3ec88483aa1c None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:56.074181 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-ce77a03e-0b12-4b45-9a29-c8adeb5033ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff2b255c631c4e8f852db18ef7bace46', 'request_id': 'req-ce77a03e-0b12-4b45-9a29-c8adeb5033ca'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:56.080074 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-ce77a03e-0b12-4b45-9a29-c8adeb5033ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff2b255c631c4e8f852db18ef7bace46', 'request_id': 'req-ce77a03e-0b12-4b45-9a29-c8adeb5033ca'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:56.088133 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-ce77a03e-0b12-4b45-9a29-c8adeb5033ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff2b255c631c4e8f852db18ef7bace46', 'request_id': 'req-ce77a03e-0b12-4b45-9a29-c8adeb5033ca'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:56.095588 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:56.100309 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-ce77a03e-0b12-4b45-9a29-c8adeb5033ca admin admin] The token cache is being invalidate because project ff2b255c631c4e8f852db18ef7bace46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:21:56.100653 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-ce77a03e-0b12-4b45-9a29-c8adeb5033ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:56.101267 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-ce77a03e-0b12-4b45-9a29-c8adeb5033ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:56.102767 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1415/2849] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:21:55 2022] DELETE /identity/v3/projects/ff2b255c631c4e8f852db18ef7bace46 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:56.108213 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6d2a6-f99c-422f-8336-d3c2020e2672 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:56.108789 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6d2a6-f99c-422f-8336-d3c2020e2672 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:56.109254 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6d2a6-f99c-422f-8336-d3c2020e2672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:56.125067 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbcc8a8-0dee-4627-8690-cb2ac95623b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:56.125677 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbcc8a8-0dee-4627-8690-cb2ac95623b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:56.125982 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbcc8a8-0dee-4627-8690-cb2ac95623b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:56.126258 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbcc8a8-0dee-4627-8690-cb2ac95623b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:56.126937 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbcc8a8-0dee-4627-8690-cb2ac95623b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:21:56.127247 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbcc8a8-0dee-4627-8690-cb2ac95623b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:56.128746 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-a3b6d2a6-f99c-422f-8336-d3c2020e2672 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:56.128872 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbcc8a8-0dee-4627-8690-cb2ac95623b4 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:56.129932 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-a3b6d2a6-f99c-422f-8336-d3c2020e2672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:56.138321 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1416/2850] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:56.143974 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1435/2851] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:21:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:21:56.147438 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6d2a6-f99c-422f-8336-d3c2020e2672 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:56.148230 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102feb4-7fdc-41a0-8589-3612151f01c3 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:56.148591 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102feb4-7fdc-41a0-8589-3612151f01c3 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:56.148809 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-a3b6d2a6-f99c-422f-8336-d3c2020e2672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:56.148999 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102feb4-7fdc-41a0-8589-3612151f01c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:56.161438 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6d2a6-f99c-422f-8336-d3c2020e2672 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:56.166419 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d77a4-2e75-4b31-b5dd-8a5f2b4ad7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:56.167051 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d77a4-2e75-4b31-b5dd-8a5f2b4ad7d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:56.167357 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d77a4-2e75-4b31-b5dd-8a5f2b4ad7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:56.167627 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d77a4-2e75-4b31-b5dd-8a5f2b4ad7d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:56.168256 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d77a4-2e75-4b31-b5dd-8a5f2b4ad7d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:56.169830 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-8102feb4-7fdc-41a0-8589-3612151f01c3 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:56.170131 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d77a4-2e75-4b31-b5dd-8a5f2b4ad7d4 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:56.171068 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-8102feb4-7fdc-41a0-8589-3612151f01c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:56.179267 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1436/2852] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:56.183439 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8102feb4-7fdc-41a0-8589-3612151f01c3 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:56.193736 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8102feb4-7fdc-41a0-8589-3612151f01c3 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:56.199372 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956324e-7b93-43c6-b4f8-fc98d0e9246c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:56.199962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956324e-7b93-43c6-b4f8-fc98d0e9246c admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:56.200239 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956324e-7b93-43c6-b4f8-fc98d0e9246c admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:56.200509 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956324e-7b93-43c6-b4f8-fc98d0e9246c admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:56.201145 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956324e-7b93-43c6-b4f8-fc98d0e9246c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:21:56.201447 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956324e-7b93-43c6-b4f8-fc98d0e9246c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:56.202270 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1417/2853] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:21:56 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:21:56.203186 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956324e-7b93-43c6-b4f8-fc98d0e9246c admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:56.206228 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d77a4-2e75-4b31-b5dd-8a5f2b4ad7d4 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:56.218870 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1437/2854] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:21:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:21:56.220835 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956324e-7b93-43c6-b4f8-fc98d0e9246c admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:56.230980 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956324e-7b93-43c6-b4f8-fc98d0e9246c admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:56.267766 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d77a4-2e75-4b31-b5dd-8a5f2b4ad7d4 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:56.272425 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a090a47a-8b48-4c98-b55a-43f0f2d742f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:56.273035 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090a47a-8b48-4c98-b55a-43f0f2d742f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:56.273342 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090a47a-8b48-4c98-b55a-43f0f2d742f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:56.273624 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090a47a-8b48-4c98-b55a-43f0f2d742f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:56.274247 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a090a47a-8b48-4c98-b55a-43f0f2d742f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:56.276188 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a090a47a-8b48-4c98-b55a-43f0f2d742f4 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:56.288366 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743866d-8237-4ec5-b4cf-0f9febf85a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:56.289021 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743866d-8237-4ec5-b4cf-0f9febf85a8a admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:56.289595 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743866d-8237-4ec5-b4cf-0f9febf85a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:56.289919 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743866d-8237-4ec5-b4cf-0f9febf85a8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:56.290939 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7743866d-8237-4ec5-b4cf-0f9febf85a8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:56.293370 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7743866d-8237-4ec5-b4cf-0f9febf85a8a admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:56.298337 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1418/2855] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:21:56 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:21:56.302350 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a090a47a-8b48-4c98-b55a-43f0f2d742f4 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:56.312428 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a090a47a-8b48-4c98-b55a-43f0f2d742f4 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:56.318207 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d156f06-f8dc-4d20-8a63-da24f2b301bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:56.318797 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d156f06-f8dc-4d20-8a63-da24f2b301bc admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:56.319108 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d156f06-f8dc-4d20-8a63-da24f2b301bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:56.319379 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d156f06-f8dc-4d20-8a63-da24f2b301bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:56.320016 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d156f06-f8dc-4d20-8a63-da24f2b301bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:56.321587 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d156f06-f8dc-4d20-8a63-da24f2b301bc admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:56.324908 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1438/2856] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:21:56 2022] POST /identity/v3/projects => generated 361 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:21:56.328868 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743866d-8237-4ec5-b4cf-0f9febf85a8a admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:56.335340 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1419/2857] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:21:56.336458 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d156f06-f8dc-4d20-8a63-da24f2b301bc admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:56.346312 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d156f06-f8dc-4d20-8a63-da24f2b301bc admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:56.396258 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:56.397044 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743866d-8237-4ec5-b4cf-0f9febf85a8a admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:56.402847 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea44de1-e81c-46dc-8bc4-6763d08d8f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:56.403477 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea44de1-e81c-46dc-8bc4-6763d08d8f6b admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:56.403752 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea44de1-e81c-46dc-8bc4-6763d08d8f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:56.404028 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea44de1-e81c-46dc-8bc4-6763d08d8f6b admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:56.404734 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea44de1-e81c-46dc-8bc4-6763d08d8f6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:56.406638 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea44de1-e81c-46dc-8bc4-6763d08d8f6b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:56.408934 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac301080-53c6-48f6-ad4d-9c4007b5c747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:56.409588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac301080-53c6-48f6-ad4d-9c4007b5c747 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:56.409866 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac301080-53c6-48f6-ad4d-9c4007b5c747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:56.410137 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac301080-53c6-48f6-ad4d-9c4007b5c747 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:56.410779 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac301080-53c6-48f6-ad4d-9c4007b5c747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:56.412375 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac301080-53c6-48f6-ad4d-9c4007b5c747 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:56.422288 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1420/2858] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:21:56.426339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac301080-53c6-48f6-ad4d-9c4007b5c747 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:56.433922 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1439/2859] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:21:56 2022] POST /identity/v3/users => generated 281 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:21:56.435702 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea44de1-e81c-46dc-8bc4-6763d08d8f6b admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:56.438676 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac301080-53c6-48f6-ad4d-9c4007b5c747 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:56.445486 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52620fac-d772-4cfb-bd94-928bdd3c7dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:56.445662 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea44de1-e81c-46dc-8bc4-6763d08d8f6b admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:56.445783 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52620fac-d772-4cfb-bd94-928bdd3c7dec admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:56.445783 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52620fac-d772-4cfb-bd94-928bdd3c7dec admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:56.445985 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52620fac-d772-4cfb-bd94-928bdd3c7dec admin admin] PATH_INFO: `/v3/projects/caed5793408e49edb40acad5410e02a9/users/c31fb669f8f14c8bbd5b7ec3450c930d/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:56.451350 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70dd6f5-07c0-4dc9-bda2-abfedf1d4e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:56.451951 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70dd6f5-07c0-4dc9-bda2-abfedf1d4e5a admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:56.452231 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70dd6f5-07c0-4dc9-bda2-abfedf1d4e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:56.452507 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70dd6f5-07c0-4dc9-bda2-abfedf1d4e5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:56.453153 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70dd6f5-07c0-4dc9-bda2-abfedf1d4e5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:56.454739 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70dd6f5-07c0-4dc9-bda2-abfedf1d4e5a admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:56.466052 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1440/2860] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:21:56.468980 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52620fac-d772-4cfb-bd94-928bdd3c7dec admin admin] RBAC: Authorizing `identity:create_grant(project_id=caed5793408e49edb40acad5410e02a9, user_id=c31fb669f8f14c8bbd5b7ec3450c930d, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:56.469694 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70dd6f5-07c0-4dc9-bda2-abfedf1d4e5a admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:56.471906 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52620fac-d772-4cfb-bd94-928bdd3c7dec admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:56.480207 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70dd6f5-07c0-4dc9-bda2-abfedf1d4e5a admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:56.485949 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1421/2861] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:21:56 2022] PUT /identity/v3/projects/caed5793408e49edb40acad5410e02a9/users/c31fb669f8f14c8bbd5b7ec3450c930d/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:56.489320 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:56.492711 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96fc353-601b-451d-b931-45ac47d1ceac None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:56.492994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96fc353-601b-451d-b931-45ac47d1ceac None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:56.493258 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96fc353-601b-451d-b931-45ac47d1ceac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:56.516529 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-c96fc353-601b-451d-b931-45ac47d1ceac None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:56.518010 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-c96fc353-601b-451d-b931-45ac47d1ceac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:56.546490 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0f7b6-c531-48ce-b4c9-15385ce42679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:56.547325 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0f7b6-c531-48ce-b4c9-15385ce42679 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:56.547595 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0f7b6-c531-48ce-b4c9-15385ce42679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:56.547954 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0f7b6-c531-48ce-b4c9-15385ce42679 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:56.548536 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab0f7b6-c531-48ce-b4c9-15385ce42679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:56.550306 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab0f7b6-c531-48ce-b4c9-15385ce42679 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:56.564887 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1441/2862] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:21:56.566313 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0f7b6-c531-48ce-b4c9-15385ce42679 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:56.583070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0f7b6-c531-48ce-b4c9-15385ce42679 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:56.583070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ba08b-9203-4a6a-a23d-14474908db2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:56.583070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ba08b-9203-4a6a-a23d-14474908db2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:56.583070 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ba08b-9203-4a6a-a23d-14474908db2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:56.583427 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ba08b-9203-4a6a-a23d-14474908db2d admin admin] PATH_INFO: `/v3/projects/d83c92e04ee046e5bef2ec2db2c4144e/users/64631ab8075b4019816281fbeb7d7f77/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:56.588499 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1422/2863] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:56.597339 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96fc353-601b-451d-b931-45ac47d1ceac None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:56.598623 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-c96fc353-601b-451d-b931-45ac47d1ceac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:56.598897 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ba08b-9203-4a6a-a23d-14474908db2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d83c92e04ee046e5bef2ec2db2c4144e, user_id=64631ab8075b4019816281fbeb7d7f77, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:56.601563 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ba08b-9203-4a6a-a23d-14474908db2d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:56.615718 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1442/2864] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:21:56 2022] PUT /identity/v3/projects/d83c92e04ee046e5bef2ec2db2c4144e/users/64631ab8075b4019816281fbeb7d7f77/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:56.626176 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f47d8-f4fa-481d-8c1e-36d4df6e35ea None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:56.626465 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f47d8-f4fa-481d-8c1e-36d4df6e35ea None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:56.626747 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f47d8-f4fa-481d-8c1e-36d4df6e35ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:56.656082 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-4a7f47d8-f4fa-481d-8c1e-36d4df6e35ea None None] MFA Rules not processed for user `64631ab8075b4019816281fbeb7d7f77`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:56.657286 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-4a7f47d8-f4fa-481d-8c1e-36d4df6e35ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:56.660076 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96fc353-601b-451d-b931-45ac47d1ceac None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:56.720393 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4b2c9-e1d9-4bf9-93dd-fefdd19e2bdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:56.721027 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4b2c9-e1d9-4bf9-93dd-fefdd19e2bdf service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:56.721287 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4b2c9-e1d9-4bf9-93dd-fefdd19e2bdf service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:56.721558 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4b2c9-e1d9-4bf9-93dd-fefdd19e2bdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:56.731654 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1443/2865] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:21:56.769345 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f47d8-f4fa-481d-8c1e-36d4df6e35ea None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:56.770919 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-4a7f47d8-f4fa-481d-8c1e-36d4df6e35ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:56.771775 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:56.813320 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb4b2c9-e1d9-4bf9-93dd-fefdd19e2bdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:56.815182 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb4b2c9-e1d9-4bf9-93dd-fefdd19e2bdf service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:56.827671 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1423/2866] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:21:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:21:56.853459 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f47d8-f4fa-481d-8c1e-36d4df6e35ea None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:56.858395 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f13a033-8cfa-4652-a7f9-2c617d6f03f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:56.859034 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13a033-8cfa-4652-a7f9-2c617d6f03f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:56.859337 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13a033-8cfa-4652-a7f9-2c617d6f03f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:56.859607 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13a033-8cfa-4652-a7f9-2c617d6f03f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:56.869550 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f13a033-8cfa-4652-a7f9-2c617d6f03f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:56.871527 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f13a033-8cfa-4652-a7f9-2c617d6f03f5 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:56.880715 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1444/2867] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:21:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:21:56.897677 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4b2c9-e1d9-4bf9-93dd-fefdd19e2bdf service neutron] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:56.908235 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4b2c9-e1d9-4bf9-93dd-fefdd19e2bdf service neutron] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:56.913572 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c4180c-2ac4-41fa-acad-7bbd03d38201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:56.914197 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4180c-2ac4-41fa-acad-7bbd03d38201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:56.914476 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4180c-2ac4-41fa-acad-7bbd03d38201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:56.914748 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4180c-2ac4-41fa-acad-7bbd03d38201 admin admin] PATH_INFO: `/v3/users/64631ab8075b4019816281fbeb7d7f77` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:56.916033 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c4180c-2ac4-41fa-acad-7bbd03d38201 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64631ab8075b4019816281fbeb7d7f77)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:56.918631 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c4180c-2ac4-41fa-acad-7bbd03d38201 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:56.970743 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-35c4180c-2ac4-41fa-acad-7bbd03d38201 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64631ab8075b4019816281fbeb7d7f77'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:56.982060 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-35c4180c-2ac4-41fa-acad-7bbd03d38201 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64631ab8075b4019816281fbeb7d7f77'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:56.983066 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-35c4180c-2ac4-41fa-acad-7bbd03d38201 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64631ab8075b4019816281fbeb7d7f77'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:56.983688 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-35c4180c-2ac4-41fa-acad-7bbd03d38201 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64631ab8075b4019816281fbeb7d7f77'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:56.991565 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-35c4180c-2ac4-41fa-acad-7bbd03d38201 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64631ab8075b4019816281fbeb7d7f77'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:57.010267 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1424/2868] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:21:56 2022] DELETE /identity/v3/users/64631ab8075b4019816281fbeb7d7f77 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:21:57.015074 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e90a9-d394-4333-80ee-522f5bc65d82 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:57.015469 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e90a9-d394-4333-80ee-522f5bc65d82 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:57.015654 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e90a9-d394-4333-80ee-522f5bc65d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:57.035900 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-493e90a9-d394-4333-80ee-522f5bc65d82 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:21:57.037295 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-493e90a9-d394-4333-80ee-522f5bc65d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:57.047921 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:21:57.101364 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1445/2869] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:21:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 14:21:57.112207 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c4180c-2ac4-41fa-acad-7bbd03d38201 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:57.113635 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-35c4180c-2ac4-41fa-acad-7bbd03d38201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:57.192172 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c4180c-2ac4-41fa-acad-7bbd03d38201 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:57.197436 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de93144-b628-451a-9958-939843ee4d27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:57.198042 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de93144-b628-451a-9958-939843ee4d27 service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:57.198322 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de93144-b628-451a-9958-939843ee4d27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:57.198591 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de93144-b628-451a-9958-939843ee4d27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:57.208815 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de93144-b628-451a-9958-939843ee4d27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:57.210237 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de93144-b628-451a-9958-939843ee4d27 service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:57.220994 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1425/2870] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:21:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:21:57.674604 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e90a9-d394-4333-80ee-522f5bc65d82 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:21:57.677151 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-493e90a9-d394-4333-80ee-522f5bc65d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:21:57.701700 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e90a9-d394-4333-80ee-522f5bc65d82 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:21:57.709907 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d754ad3-e8b4-40c1-a7ce-6fbabb7f82f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:21:57.710715 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d754ad3-e8b4-40c1-a7ce-6fbabb7f82f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:21:57.711028 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d754ad3-e8b4-40c1-a7ce-6fbabb7f82f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:21:57.711302 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d754ad3-e8b4-40c1-a7ce-6fbabb7f82f7 admin admin] PATH_INFO: `/v3/projects/d83c92e04ee046e5bef2ec2db2c4144e` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:21:57.712945 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d754ad3-e8b4-40c1-a7ce-6fbabb7f82f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d83c92e04ee046e5bef2ec2db2c4144e)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:21:57.715761 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d754ad3-e8b4-40c1-a7ce-6fbabb7f82f7 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:21:57.799919 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7d754ad3-e8b4-40c1-a7ce-6fbabb7f82f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd83c92e04ee046e5bef2ec2db2c4144e', 'request_id': 'req-7d754ad3-e8b4-40c1-a7ce-6fbabb7f82f7'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:57.813388 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7d754ad3-e8b4-40c1-a7ce-6fbabb7f82f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd83c92e04ee046e5bef2ec2db2c4144e', 'request_id': 'req-7d754ad3-e8b4-40c1-a7ce-6fbabb7f82f7'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:57.824121 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7d754ad3-e8b4-40c1-a7ce-6fbabb7f82f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd83c92e04ee046e5bef2ec2db2c4144e', 'request_id': 'req-7d754ad3-e8b4-40c1-a7ce-6fbabb7f82f7'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:57.852934 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7d754ad3-e8b4-40c1-a7ce-6fbabb7f82f7 admin admin] The token cache is being invalidate because project d83c92e04ee046e5bef2ec2db2c4144e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:21:57.853626 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7d754ad3-e8b4-40c1-a7ce-6fbabb7f82f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:57.854260 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7d754ad3-e8b4-40c1-a7ce-6fbabb7f82f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:21:57.862888 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1446/2871] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:21:57 2022] DELETE /identity/v3/projects/d83c92e04ee046e5bef2ec2db2c4144e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:22:01.188928 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf75278-9766-4990-a8fe-af437103f316 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:01.189447 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf75278-9766-4990-a8fe-af437103f316 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:01.189447 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf75278-9766-4990-a8fe-af437103f316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:01.218774 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-7bf75278-9766-4990-a8fe-af437103f316 None None] MFA Rules not processed for user `c31fb669f8f14c8bbd5b7ec3450c930d`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:22:01.220008 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-7bf75278-9766-4990-a8fe-af437103f316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:22:01.253205 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:22:01.288050 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1426/2872] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:22:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:22:01.292584 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d754ad3-e8b4-40c1-a7ce-6fbabb7f82f7 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:22:01.294048 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7d754ad3-e8b4-40c1-a7ce-6fbabb7f82f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:22:01.361786 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d754ad3-e8b4-40c1-a7ce-6fbabb7f82f7 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:22:01.366408 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c6f87-4635-4281-874a-f33cddfe5a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:22:01.367055 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c6f87-4635-4281-874a-f33cddfe5a21 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:01.367341 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c6f87-4635-4281-874a-f33cddfe5a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:01.367616 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c6f87-4635-4281-874a-f33cddfe5a21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:01.368251 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c6f87-4635-4281-874a-f33cddfe5a21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:22:01.370697 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c6f87-4635-4281-874a-f33cddfe5a21 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:22:01.400467 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1447/2873] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:22:01 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:22:01.404522 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf75278-9766-4990-a8fe-af437103f316 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:22:01.416252 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf75278-9766-4990-a8fe-af437103f316 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:22:01.487498 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d8c5d-81f1-4316-94d2-c709ad16f2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:22:01.488111 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d8c5d-81f1-4316-94d2-c709ad16f2f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:01.488389 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d8c5d-81f1-4316-94d2-c709ad16f2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:01.488659 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d8c5d-81f1-4316-94d2-c709ad16f2f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:01.489300 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0d8c5d-81f1-4316-94d2-c709ad16f2f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:22:01.491233 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0d8c5d-81f1-4316-94d2-c709ad16f2f2 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:22:01.512952 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1427/2874] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:22:01 2022] POST /identity/v3/users => generated 296 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:22:01.517063 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c6f87-4635-4281-874a-f33cddfe5a21 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:22:01.527210 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c6f87-4635-4281-874a-f33cddfe5a21 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:22:01.531720 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e4397-77a2-43b6-b7ed-df3321845238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:22:01.532311 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e4397-77a2-43b6-b7ed-df3321845238 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:01.532592 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e4397-77a2-43b6-b7ed-df3321845238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:01.532863 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e4397-77a2-43b6-b7ed-df3321845238 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:01.533510 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e4397-77a2-43b6-b7ed-df3321845238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:22:01.535110 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e4397-77a2-43b6-b7ed-df3321845238 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:22:01.547112 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1448/2875] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:22:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:22:01.549701 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d8c5d-81f1-4316-94d2-c709ad16f2f2 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:22:01.560388 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d8c5d-81f1-4316-94d2-c709ad16f2f2 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:22:01.565125 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8055f5-479a-437d-8272-955ba349f144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:22:01.565768 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8055f5-479a-437d-8272-955ba349f144 admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:01.566042 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8055f5-479a-437d-8272-955ba349f144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:01.566308 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8055f5-479a-437d-8272-955ba349f144 admin admin] PATH_INFO: `/v3/projects/438b10e90ef242a9a61e81d427823e44/users/a9a497f1250e4fc681892928a7479fc2/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:01.582109 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8055f5-479a-437d-8272-955ba349f144 admin admin] RBAC: Authorizing `identity:create_grant(project_id=438b10e90ef242a9a61e81d427823e44, user_id=a9a497f1250e4fc681892928a7479fc2, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:22:01.584927 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8055f5-479a-437d-8272-955ba349f144 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:22:01.598789 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1428/2876] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:22:01 2022] PUT /identity/v3/projects/438b10e90ef242a9a61e81d427823e44/users/a9a497f1250e4fc681892928a7479fc2/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:22:01.602911 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e4397-77a2-43b6-b7ed-df3321845238 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:22:01.613736 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e4397-77a2-43b6-b7ed-df3321845238 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:22:01.623229 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:22:01.672402 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5aaf4-b666-4108-bdd4-ad308974f372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:22:01.673009 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5aaf4-b666-4108-bdd4-ad308974f372 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:01.673738 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5aaf4-b666-4108-bdd4-ad308974f372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:01.673738 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5aaf4-b666-4108-bdd4-ad308974f372 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:01.674193 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5aaf4-b666-4108-bdd4-ad308974f372 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:22:01.675802 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5aaf4-b666-4108-bdd4-ad308974f372 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:22:01.687279 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1449/2877] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:22:01 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:22:01.691043 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8055f5-479a-437d-8272-955ba349f144 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:22:01.701447 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8055f5-479a-437d-8272-955ba349f144 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:22:01.706130 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d426e-dd1a-4e55-afc5-9622f6666c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:22:01.706770 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d426e-dd1a-4e55-afc5-9622f6666c6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:01.707066 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d426e-dd1a-4e55-afc5-9622f6666c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:01.707331 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d426e-dd1a-4e55-afc5-9622f6666c6f admin admin] PATH_INFO: `/v3/projects/438b10e90ef242a9a61e81d427823e44/users/a9a497f1250e4fc681892928a7479fc2/roles/e14564f00daf48a28d4e32da8cd2efad` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:01.709200 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d426e-dd1a-4e55-afc5-9622f6666c6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=438b10e90ef242a9a61e81d427823e44, user_id=a9a497f1250e4fc681892928a7479fc2, role_id=e14564f00daf48a28d4e32da8cd2efad)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:22:01.713475 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d426e-dd1a-4e55-afc5-9622f6666c6f admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:22:01.727453 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1429/2878] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:22:01 2022] PUT /identity/v3/projects/438b10e90ef242a9a61e81d427823e44/users/a9a497f1250e4fc681892928a7479fc2/roles/e14564f00daf48a28d4e32da8cd2efad => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:22:06.426269 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74dc53-8711-40e5-a2db-1a77a0a3a6fd None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:06.426793 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74dc53-8711-40e5-a2db-1a77a0a3a6fd None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:06.426793 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74dc53-8711-40e5-a2db-1a77a0a3a6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:06.458098 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-4c74dc53-8711-40e5-a2db-1a77a0a3a6fd None None] MFA Rules not processed for user `a9a497f1250e4fc681892928a7479fc2`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:22:06.459377 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-4c74dc53-8711-40e5-a2db-1a77a0a3a6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:22:06.548957 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1450/2879] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:22:06 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:22:06.569597 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d426e-dd1a-4e55-afc5-9622f6666c6f admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:22:06.571394 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-db3d426e-dd1a-4e55-afc5-9622f6666c6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:22:06.690440 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:22:06.694649 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d426e-dd1a-4e55-afc5-9622f6666c6f admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:22:06.702319 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce1447b-b016-46bc-9699-76c34011f588 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:22:06.703053 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1447b-b016-46bc-9699-76c34011f588 service nova] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:06.703322 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1447b-b016-46bc-9699-76c34011f588 service nova] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:06.703830 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1447b-b016-46bc-9699-76c34011f588 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:06.765217 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce1447b-b016-46bc-9699-76c34011f588 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:22:06.766986 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce1447b-b016-46bc-9699-76c34011f588 service nova] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:22:06.777366 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1430/2880] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:22:06 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:22:16.545064 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=57 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:22:16.773027 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=26 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:22:31.633041 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109bc7d-2bae-49e1-9087-138c732d3135 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:31.633610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109bc7d-2bae-49e1-9087-138c732d3135 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:31.633610 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109bc7d-2bae-49e1-9087-138c732d3135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:31.652325 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-0109bc7d-2bae-49e1-9087-138c732d3135 None None] MFA Rules not processed for user `c31fb669f8f14c8bbd5b7ec3450c930d`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:22:31.653899 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-0109bc7d-2bae-49e1-9087-138c732d3135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:22:31.662031 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1451/2881] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:22:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:22:35.745917 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce1447b-b016-46bc-9699-76c34011f588 service nova] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:22:35.747461 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-cce1447b-b016-46bc-9699-76c34011f588 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:22:35.818461 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce1447b-b016-46bc-9699-76c34011f588 service nova] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:22:35.823249 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891d54c-e335-4b46-acb0-b18338187cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:22:35.823906 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891d54c-e335-4b46-acb0-b18338187cf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:35.824184 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891d54c-e335-4b46-acb0-b18338187cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:35.824455 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891d54c-e335-4b46-acb0-b18338187cf0 admin admin] PATH_INFO: `/v3/users/63ef31be1a4f4bf5b6a02acd42d68ee8` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:35.825602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891d54c-e335-4b46-acb0-b18338187cf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63ef31be1a4f4bf5b6a02acd42d68ee8)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:22:35.828481 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891d54c-e335-4b46-acb0-b18338187cf0 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:22:35.884209 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-6891d54c-e335-4b46-acb0-b18338187cf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63ef31be1a4f4bf5b6a02acd42d68ee8'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:22:35.897385 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-6891d54c-e335-4b46-acb0-b18338187cf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63ef31be1a4f4bf5b6a02acd42d68ee8'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:22:35.898077 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-6891d54c-e335-4b46-acb0-b18338187cf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63ef31be1a4f4bf5b6a02acd42d68ee8'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:22:35.898697 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-6891d54c-e335-4b46-acb0-b18338187cf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63ef31be1a4f4bf5b6a02acd42d68ee8'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:22:35.907486 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-6891d54c-e335-4b46-acb0-b18338187cf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63ef31be1a4f4bf5b6a02acd42d68ee8'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:22:35.928507 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1431/2882] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:22:35 2022] DELETE /identity/v3/users/63ef31be1a4f4bf5b6a02acd42d68ee8 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:22:35.932962 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ed2e5-b3b8-4957-8b84-096ab7b08e0e None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:35.933456 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ed2e5-b3b8-4957-8b84-096ab7b08e0e None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:35.933559 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ed2e5-b3b8-4957-8b84-096ab7b08e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:35.954104 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-185ed2e5-b3b8-4957-8b84-096ab7b08e0e None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:22:35.955377 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-185ed2e5-b3b8-4957-8b84-096ab7b08e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:22:36.027535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1452/2883] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:22:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:22:36.037814 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891d54c-e335-4b46-acb0-b18338187cf0 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:22:36.039260 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-6891d54c-e335-4b46-acb0-b18338187cf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:22:36.099285 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891d54c-e335-4b46-acb0-b18338187cf0 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:22:36.103059 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a58c19e-8cc5-4f97-b2f2-c246af169cc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:22:36.103671 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58c19e-8cc5-4f97-b2f2-c246af169cc5 service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:36.103953 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58c19e-8cc5-4f97-b2f2-c246af169cc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:36.104226 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58c19e-8cc5-4f97-b2f2-c246af169cc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:36.114953 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a58c19e-8cc5-4f97-b2f2-c246af169cc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:22:36.116421 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a58c19e-8cc5-4f97-b2f2-c246af169cc5 service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:22:36.126594 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1432/2884] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:22:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:22:36.476726 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ed2e5-b3b8-4957-8b84-096ab7b08e0e None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:22:36.478485 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-185ed2e5-b3b8-4957-8b84-096ab7b08e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:22:36.491546 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ed2e5-b3b8-4957-8b84-096ab7b08e0e None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:22:36.496335 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b084ebc-e65a-4cf9-a980-123c49d931de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:22:36.496972 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b084ebc-e65a-4cf9-a980-123c49d931de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:36.497253 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b084ebc-e65a-4cf9-a980-123c49d931de admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:36.497530 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b084ebc-e65a-4cf9-a980-123c49d931de admin admin] PATH_INFO: `/v3/projects/efc2b68289774509abef904add3762d1` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:36.498660 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b084ebc-e65a-4cf9-a980-123c49d931de admin admin] RBAC: Authorizing `identity:delete_project(project_id=efc2b68289774509abef904add3762d1)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:22:36.501165 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b084ebc-e65a-4cf9-a980-123c49d931de admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:22:36.547431 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-8b084ebc-e65a-4cf9-a980-123c49d931de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efc2b68289774509abef904add3762d1', 'request_id': 'req-8b084ebc-e65a-4cf9-a980-123c49d931de'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:22:36.554478 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-8b084ebc-e65a-4cf9-a980-123c49d931de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efc2b68289774509abef904add3762d1', 'request_id': 'req-8b084ebc-e65a-4cf9-a980-123c49d931de'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:22:36.560296 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-8b084ebc-e65a-4cf9-a980-123c49d931de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efc2b68289774509abef904add3762d1', 'request_id': 'req-8b084ebc-e65a-4cf9-a980-123c49d931de'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:22:36.571716 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-8b084ebc-e65a-4cf9-a980-123c49d931de admin admin] The token cache is being invalidate because project efc2b68289774509abef904add3762d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:22:36.572005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-8b084ebc-e65a-4cf9-a980-123c49d931de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:22:36.572629 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-8b084ebc-e65a-4cf9-a980-123c49d931de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:22:36.574087 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1453/2885] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:22:36 2022] DELETE /identity/v3/projects/efc2b68289774509abef904add3762d1 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:22:36.579177 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12a721-f372-473d-83e5-96c4c4a54a78 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:36.579467 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12a721-f372-473d-83e5-96c4c4a54a78 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:36.579740 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12a721-f372-473d-83e5-96c4c4a54a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:36.598765 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-9f12a721-f372-473d-83e5-96c4c4a54a78 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:22:36.599985 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-9f12a721-f372-473d-83e5-96c4c4a54a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:22:36.616852 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:22:36.664695 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1433/2886] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:22:36.673199 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b314e-30e6-4d7d-82d2-5c3075cd193b None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:36.673484 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b314e-30e6-4d7d-82d2-5c3075cd193b None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:36.673757 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b314e-30e6-4d7d-82d2-5c3075cd193b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:36.694327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-214b314e-30e6-4d7d-82d2-5c3075cd193b None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:22:36.695562 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-214b314e-30e6-4d7d-82d2-5c3075cd193b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:22:36.703453 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1454/2887] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:22:36.707661 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12a721-f372-473d-83e5-96c4c4a54a78 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:22:36.717424 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12a721-f372-473d-83e5-96c4c4a54a78 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:22:36.722129 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec376ca-84aa-48fd-a22c-9fe93a6f59a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:22:36.722686 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec376ca-84aa-48fd-a22c-9fe93a6f59a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:36.723167 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec376ca-84aa-48fd-a22c-9fe93a6f59a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:36.723439 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec376ca-84aa-48fd-a22c-9fe93a6f59a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:36.724077 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec376ca-84aa-48fd-a22c-9fe93a6f59a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:22:36.724372 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec376ca-84aa-48fd-a22c-9fe93a6f59a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:22:36.725945 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec376ca-84aa-48fd-a22c-9fe93a6f59a8 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:22:36.737698 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1434/2888] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:22:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 14:22:36.742056 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b314e-30e6-4d7d-82d2-5c3075cd193b None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:22:36.751680 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b314e-30e6-4d7d-82d2-5c3075cd193b None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:22:36.756216 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7768ac-a5ce-4c33-9b24-acf35ad350ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:22:36.756798 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7768ac-a5ce-4c33-9b24-acf35ad350ba admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:36.757073 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7768ac-a5ce-4c33-9b24-acf35ad350ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:36.757340 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7768ac-a5ce-4c33-9b24-acf35ad350ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:36.757991 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7768ac-a5ce-4c33-9b24-acf35ad350ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:22:36.759894 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7768ac-a5ce-4c33-9b24-acf35ad350ba admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:22:36.785915 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1455/2889] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:22:36 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:22:36.790023 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec376ca-84aa-48fd-a22c-9fe93a6f59a8 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:22:36.800710 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec376ca-84aa-48fd-a22c-9fe93a6f59a8 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:22:36.856769 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffdfb75-9b0a-420e-8e7c-146d69a2c452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:22:36.857371 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffdfb75-9b0a-420e-8e7c-146d69a2c452 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:36.857653 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffdfb75-9b0a-420e-8e7c-146d69a2c452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:36.857926 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffdfb75-9b0a-420e-8e7c-146d69a2c452 admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:36.859095 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffdfb75-9b0a-420e-8e7c-146d69a2c452 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:22:36.860999 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffdfb75-9b0a-420e-8e7c-146d69a2c452 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:22:36.882306 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1435/2890] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:22:36 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:22:36.886647 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7768ac-a5ce-4c33-9b24-acf35ad350ba admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:22:36.896900 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7768ac-a5ce-4c33-9b24-acf35ad350ba admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:22:36.901535 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06894f07-a21b-4baa-8ad5-6193dd99a038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:22:36.902128 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06894f07-a21b-4baa-8ad5-6193dd99a038 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:36.902408 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06894f07-a21b-4baa-8ad5-6193dd99a038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:36.902679 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06894f07-a21b-4baa-8ad5-6193dd99a038 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:36.903363 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06894f07-a21b-4baa-8ad5-6193dd99a038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:22:36.904928 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06894f07-a21b-4baa-8ad5-6193dd99a038 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:22:36.915153 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1456/2891] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:22:36.918934 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffdfb75-9b0a-420e-8e7c-146d69a2c452 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:22:36.929323 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffdfb75-9b0a-420e-8e7c-146d69a2c452 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:22:36.933918 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5b693-8597-432f-aec0-e63eb5a745f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:22:36.934763 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5b693-8597-432f-aec0-e63eb5a745f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:36.935076 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5b693-8597-432f-aec0-e63eb5a745f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:36.935414 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5b693-8597-432f-aec0-e63eb5a745f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:36.936087 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5b693-8597-432f-aec0-e63eb5a745f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:22:36.937690 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5b693-8597-432f-aec0-e63eb5a745f9 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:22:36.948023 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1436/2892] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:22:36.951947 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06894f07-a21b-4baa-8ad5-6193dd99a038 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:22:36.961613 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06894f07-a21b-4baa-8ad5-6193dd99a038 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:22:36.966156 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d87fada-4521-4643-beb4-fdea7fe0c45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:22:36.966783 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87fada-4521-4643-beb4-fdea7fe0c45f admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:36.967093 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87fada-4521-4643-beb4-fdea7fe0c45f admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:36.967368 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87fada-4521-4643-beb4-fdea7fe0c45f admin admin] PATH_INFO: `/v3/projects/deea94b380ad47c3a107eb2224bd3edd/users/3c758f73c47d443f8be87cecc0cf83c3/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:36.982129 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d87fada-4521-4643-beb4-fdea7fe0c45f admin admin] RBAC: Authorizing `identity:create_grant(project_id=deea94b380ad47c3a107eb2224bd3edd, user_id=3c758f73c47d443f8be87cecc0cf83c3, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:22:36.985537 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d87fada-4521-4643-beb4-fdea7fe0c45f admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:22:36.998243 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1457/2893] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:22:36 2022] PUT /identity/v3/projects/deea94b380ad47c3a107eb2224bd3edd/users/3c758f73c47d443f8be87cecc0cf83c3/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:22:37.065536 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36aa604-bbdb-4970-a51b-16c0572717db None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:37.065824 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36aa604-bbdb-4970-a51b-16c0572717db None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:37.066094 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36aa604-bbdb-4970-a51b-16c0572717db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:37.095856 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-b36aa604-bbdb-4970-a51b-16c0572717db None None] MFA Rules not processed for user `3c758f73c47d443f8be87cecc0cf83c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:22:37.097801 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-b36aa604-bbdb-4970-a51b-16c0572717db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:22:37.165790 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1437/2894] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:22:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:22:37.176961 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d87fada-4521-4643-beb4-fdea7fe0c45f admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:22:37.178680 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-9d87fada-4521-4643-beb4-fdea7fe0c45f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:22:37.193810 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:22:37.238688 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d87fada-4521-4643-beb4-fdea7fe0c45f admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:22:37.242236 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c49159-3085-429b-8ed7-159f7572b60a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:22:37.242830 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c49159-3085-429b-8ed7-159f7572b60a service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:37.243136 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c49159-3085-429b-8ed7-159f7572b60a service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:37.243406 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c49159-3085-429b-8ed7-159f7572b60a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:37.251440 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c49159-3085-429b-8ed7-159f7572b60a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:22:37.252848 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c49159-3085-429b-8ed7-159f7572b60a service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:22:37.261310 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1458/2895] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:22:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:22:47.163047 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:22:47.214470 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36aa604-bbdb-4970-a51b-16c0572717db None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:22:47.257415 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=20 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:22:47.282047 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36aa604-bbdb-4970-a51b-16c0572717db None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:22:47.286604 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb5bc2-9efa-4266-84f3-09b09ee6da79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:22:47.287273 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb5bc2-9efa-4266-84f3-09b09ee6da79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:47.288171 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb5bc2-9efa-4266-84f3-09b09ee6da79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:47.288461 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb5bc2-9efa-4266-84f3-09b09ee6da79 admin admin] PATH_INFO: `/v3/users/3c758f73c47d443f8be87cecc0cf83c3` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:47.289592 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cb5bc2-9efa-4266-84f3-09b09ee6da79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c758f73c47d443f8be87cecc0cf83c3)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:22:47.292150 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cb5bc2-9efa-4266-84f3-09b09ee6da79 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:22:47.347787 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-a3cb5bc2-9efa-4266-84f3-09b09ee6da79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c758f73c47d443f8be87cecc0cf83c3'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:22:47.359839 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-a3cb5bc2-9efa-4266-84f3-09b09ee6da79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c758f73c47d443f8be87cecc0cf83c3'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:22:47.361002 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-a3cb5bc2-9efa-4266-84f3-09b09ee6da79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c758f73c47d443f8be87cecc0cf83c3'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:22:47.361672 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-a3cb5bc2-9efa-4266-84f3-09b09ee6da79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c758f73c47d443f8be87cecc0cf83c3'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:22:47.369218 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-a3cb5bc2-9efa-4266-84f3-09b09ee6da79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c758f73c47d443f8be87cecc0cf83c3'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:22:47.388512 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1438/2896] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:22:47 2022] DELETE /identity/v3/users/3c758f73c47d443f8be87cecc0cf83c3 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:22:47.393455 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a68236-9a9a-428c-bd1d-81d53aa9d536 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:47.393740 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a68236-9a9a-428c-bd1d-81d53aa9d536 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:47.394008 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a68236-9a9a-428c-bd1d-81d53aa9d536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:47.413343 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-d8a68236-9a9a-428c-bd1d-81d53aa9d536 None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:22:47.414600 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d8a68236-9a9a-428c-bd1d-81d53aa9d536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:22:47.481307 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1459/2897] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:22:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:22:47.492985 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb5bc2-9efa-4266-84f3-09b09ee6da79 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:22:47.494386 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-a3cb5bc2-9efa-4266-84f3-09b09ee6da79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:22:47.554755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb5bc2-9efa-4266-84f3-09b09ee6da79 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:22:47.558627 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59b34f-1026-4931-bd18-c65534e7c5bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:22:47.559533 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59b34f-1026-4931-bd18-c65534e7c5bc service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:47.559810 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59b34f-1026-4931-bd18-c65534e7c5bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:47.560081 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59b34f-1026-4931-bd18-c65534e7c5bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:47.569808 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d59b34f-1026-4931-bd18-c65534e7c5bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:22:47.571265 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d59b34f-1026-4931-bd18-c65534e7c5bc service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:22:47.581164 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1439/2898] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:22:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:22:47.908166 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a68236-9a9a-428c-bd1d-81d53aa9d536 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:22:47.910295 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d8a68236-9a9a-428c-bd1d-81d53aa9d536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:22:47.931154 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a68236-9a9a-428c-bd1d-81d53aa9d536 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:22:47.937837 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd8aec5-bf98-451b-86b5-4fe2c192e564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:22:47.938410 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8aec5-bf98-451b-86b5-4fe2c192e564 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:47.938749 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8aec5-bf98-451b-86b5-4fe2c192e564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:47.939120 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8aec5-bf98-451b-86b5-4fe2c192e564 admin admin] PATH_INFO: `/v3/projects/deea94b380ad47c3a107eb2224bd3edd` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:47.940461 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd8aec5-bf98-451b-86b5-4fe2c192e564 admin admin] RBAC: Authorizing `identity:delete_project(project_id=deea94b380ad47c3a107eb2224bd3edd)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:22:47.942430 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd8aec5-bf98-451b-86b5-4fe2c192e564 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:22:47.997104 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-3cd8aec5-bf98-451b-86b5-4fe2c192e564 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'deea94b380ad47c3a107eb2224bd3edd', 'request_id': 'req-3cd8aec5-bf98-451b-86b5-4fe2c192e564'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:22:48.006561 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-3cd8aec5-bf98-451b-86b5-4fe2c192e564 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'deea94b380ad47c3a107eb2224bd3edd', 'request_id': 'req-3cd8aec5-bf98-451b-86b5-4fe2c192e564'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:22:48.013166 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-3cd8aec5-bf98-451b-86b5-4fe2c192e564 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'deea94b380ad47c3a107eb2224bd3edd', 'request_id': 'req-3cd8aec5-bf98-451b-86b5-4fe2c192e564'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:22:48.028197 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-3cd8aec5-bf98-451b-86b5-4fe2c192e564 admin admin] The token cache is being invalidate because project deea94b380ad47c3a107eb2224bd3edd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:22:48.028586 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-3cd8aec5-bf98-451b-86b5-4fe2c192e564 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:22:48.029381 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-3cd8aec5-bf98-451b-86b5-4fe2c192e564 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:22:48.031286 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1460/2899] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:22:47 2022] DELETE /identity/v3/projects/deea94b380ad47c3a107eb2224bd3edd => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:22:48.036390 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ad083-cc55-4f7f-b44c-b7482037941f None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:48.036876 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ad083-cc55-4f7f-b44c-b7482037941f None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:48.037154 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ad083-cc55-4f7f-b44c-b7482037941f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:48.063832 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-ec8ad083-cc55-4f7f-b44c-b7482037941f None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:22:48.065067 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-ec8ad083-cc55-4f7f-b44c-b7482037941f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:22:48.084475 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:22:48.134820 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1440/2900] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:22:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:22:48.142831 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd41285-823c-45cd-95f7-842b94c2995c None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:48.143145 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd41285-823c-45cd-95f7-842b94c2995c None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:48.143417 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd41285-823c-45cd-95f7-842b94c2995c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:48.163556 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-2dd41285-823c-45cd-95f7-842b94c2995c None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:22:48.164751 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-2dd41285-823c-45cd-95f7-842b94c2995c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:22:48.173204 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1461/2901] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:22:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:22:48.177327 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8ad083-cc55-4f7f-b44c-b7482037941f None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:22:48.188929 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8ad083-cc55-4f7f-b44c-b7482037941f None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:22:48.193711 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d978791-9a51-4d32-9df1-5a2af7b0cb67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:22:48.194312 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d978791-9a51-4d32-9df1-5a2af7b0cb67 admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:48.194595 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d978791-9a51-4d32-9df1-5a2af7b0cb67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:48.194932 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d978791-9a51-4d32-9df1-5a2af7b0cb67 admin admin] PATH_INFO: `/v3/domains` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:48.195601 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d978791-9a51-4d32-9df1-5a2af7b0cb67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81596) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 14:22:48.195906 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d978791-9a51-4d32-9df1-5a2af7b0cb67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:22:48.197492 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d978791-9a51-4d32-9df1-5a2af7b0cb67 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:22:48.208979 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1441/2902] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 12 14:22:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 14:22:48.213003 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd41285-823c-45cd-95f7-842b94c2995c None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:22:48.222558 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd41285-823c-45cd-95f7-842b94c2995c None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:22:48.227125 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82db5c-5100-4908-a657-8509b2f6fa4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:22:48.227713 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82db5c-5100-4908-a657-8509b2f6fa4a admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:48.227992 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82db5c-5100-4908-a657-8509b2f6fa4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:48.228260 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82db5c-5100-4908-a657-8509b2f6fa4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:48.228890 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e82db5c-5100-4908-a657-8509b2f6fa4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:22:48.230727 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e82db5c-5100-4908-a657-8509b2f6fa4a admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:22:48.257475 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1462/2903] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 12 14:22:48 2022] POST /identity/v3/projects => generated 397 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 14:22:48.261254 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d978791-9a51-4d32-9df1-5a2af7b0cb67 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:22:48.271272 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d978791-9a51-4d32-9df1-5a2af7b0cb67 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:22:48.326808 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc55c8-fe3a-4af8-bd3f-409db265d319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:22:48.327434 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc55c8-fe3a-4af8-bd3f-409db265d319 admin admin] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:48.327750 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc55c8-fe3a-4af8-bd3f-409db265d319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:48.328025 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc55c8-fe3a-4af8-bd3f-409db265d319 admin admin] PATH_INFO: `/v3/users` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:48.328661 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc55c8-fe3a-4af8-bd3f-409db265d319 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:22:48.330553 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc55c8-fe3a-4af8-bd3f-409db265d319 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:22:48.352367 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1442/2904] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 12 14:22:48 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 14:22:48.356461 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82db5c-5100-4908-a657-8509b2f6fa4a admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:22:48.367199 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82db5c-5100-4908-a657-8509b2f6fa4a admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:22:48.371700 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a51ab3-78a7-45d6-9723-8e351f839190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:22:48.372318 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a51ab3-78a7-45d6-9723-8e351f839190 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:48.372701 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a51ab3-78a7-45d6-9723-8e351f839190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:48.373014 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a51ab3-78a7-45d6-9723-8e351f839190 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:48.373629 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a51ab3-78a7-45d6-9723-8e351f839190 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:22:48.375263 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a51ab3-78a7-45d6-9723-8e351f839190 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:22:48.385091 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1463/2905] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:22:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 14:22:48.388906 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc55c8-fe3a-4af8-bd3f-409db265d319 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:22:48.399190 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc55c8-fe3a-4af8-bd3f-409db265d319 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:22:48.403851 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a081d967-4bb8-4e9c-b0f6-3f4bfaa31f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:22:48.404457 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a081d967-4bb8-4e9c-b0f6-3f4bfaa31f8b admin admin] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:48.404738 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a081d967-4bb8-4e9c-b0f6-3f4bfaa31f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:48.405031 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a081d967-4bb8-4e9c-b0f6-3f4bfaa31f8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:48.406186 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a081d967-4bb8-4e9c-b0f6-3f4bfaa31f8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:22:48.407845 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a081d967-4bb8-4e9c-b0f6-3f4bfaa31f8b admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:22:48.418518 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1443/2906] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 12 14:22:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 14:22:48.423455 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a51ab3-78a7-45d6-9723-8e351f839190 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:22:48.433130 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a51ab3-78a7-45d6-9723-8e351f839190 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:22:48.438010 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bfee46-1b6e-4295-8008-a7a80e811f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:22:48.438804 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bfee46-1b6e-4295-8008-a7a80e811f55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:48.439199 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bfee46-1b6e-4295-8008-a7a80e811f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:48.439470 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bfee46-1b6e-4295-8008-a7a80e811f55 admin admin] PATH_INFO: `/v3/projects/cb3e6765ec6743efa823b1c814cd5daf/users/8bf07e2d602d4323a160a446ddbc0d2e/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:48.454333 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bfee46-1b6e-4295-8008-a7a80e811f55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb3e6765ec6743efa823b1c814cd5daf, user_id=8bf07e2d602d4323a160a446ddbc0d2e, role_id=d0a6f7c5c34a4925b9efb7dd4ac915fc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:22:48.456950 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bfee46-1b6e-4295-8008-a7a80e811f55 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:22:48.471875 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1464/2907] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 12 14:22:48 2022] PUT /identity/v3/projects/cb3e6765ec6743efa823b1c814cd5daf/users/8bf07e2d602d4323a160a446ddbc0d2e/roles/d0a6f7c5c34a4925b9efb7dd4ac915fc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:22:48.483728 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda7ecb-2ba3-433b-9123-d1c43cb6ae71 None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:48.484699 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda7ecb-2ba3-433b-9123-d1c43cb6ae71 None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:48.484977 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda7ecb-2ba3-433b-9123-d1c43cb6ae71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:48.523706 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-4cda7ecb-2ba3-433b-9123-d1c43cb6ae71 None None] MFA Rules not processed for user `8bf07e2d602d4323a160a446ddbc0d2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:22:48.525172 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-4cda7ecb-2ba3-433b-9123-d1c43cb6ae71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:22:48.594566 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1444/2908] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:22:48 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:22:48.606016 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bfee46-1b6e-4295-8008-a7a80e811f55 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:22:48.608341 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f9bfee46-1b6e-4295-8008-a7a80e811f55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:22:48.631616 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:22:48.670308 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bfee46-1b6e-4295-8008-a7a80e811f55 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:22:48.673989 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad033bb-b98a-4a58-973e-67fed82fa64a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da38bc8dd2f4cf798649ce96097a4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:22:48.674596 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad033bb-b98a-4a58-973e-67fed82fa64a service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:48.674911 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad033bb-b98a-4a58-973e-67fed82fa64a service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:48.675185 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad033bb-b98a-4a58-973e-67fed82fa64a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:48.683453 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad033bb-b98a-4a58-973e-67fed82fa64a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:22:48.685135 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad033bb-b98a-4a58-973e-67fed82fa64a service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:22:48.693852 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1465/2909] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:22:48 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:22:49.215347 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda7ecb-2ba3-433b-9123-d1c43cb6ae71 None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:22:49.216945 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-4cda7ecb-2ba3-433b-9123-d1c43cb6ae71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:22:49.237884 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:22:49.280263 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda7ecb-2ba3-433b-9123-d1c43cb6ae71 None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:22:49.284256 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5525f8-6016-41af-bd34-67d9a1665dec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:22:49.285540 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5525f8-6016-41af-bd34-67d9a1665dec service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:49.285772 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5525f8-6016-41af-bd34-67d9a1665dec service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:49.286132 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5525f8-6016-41af-bd34-67d9a1665dec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:49.288724 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-db5525f8-6016-41af-bd34-67d9a1665dec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:22:49.370820 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5525f8-6016-41af-bd34-67d9a1665dec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:22:49.372314 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5525f8-6016-41af-bd34-67d9a1665dec service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:22:49.382774 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1445/2910] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:22:49 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 14:22:56.121168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad033bb-b98a-4a58-973e-67fed82fa64a service cinder] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:22:56.182306 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad033bb-b98a-4a58-973e-67fed82fa64a service cinder] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:22:56.187048 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb9610-647d-47e5-ae06-048b314a15e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:22:56.187780 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb9610-647d-47e5-ae06-048b314a15e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:56.188065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb9610-647d-47e5-ae06-048b314a15e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:56.188339 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb9610-647d-47e5-ae06-048b314a15e5 admin admin] PATH_INFO: `/v3/users/8bf07e2d602d4323a160a446ddbc0d2e` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:56.189490 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bb9610-647d-47e5-ae06-048b314a15e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bf07e2d602d4323a160a446ddbc0d2e)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:22:56.191433 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bb9610-647d-47e5-ae06-048b314a15e5 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:22:56.237036 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-12bb9610-647d-47e5-ae06-048b314a15e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bf07e2d602d4323a160a446ddbc0d2e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:22:56.246403 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-12bb9610-647d-47e5-ae06-048b314a15e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bf07e2d602d4323a160a446ddbc0d2e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:22:56.247053 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-12bb9610-647d-47e5-ae06-048b314a15e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bf07e2d602d4323a160a446ddbc0d2e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:22:56.247706 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-12bb9610-647d-47e5-ae06-048b314a15e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bf07e2d602d4323a160a446ddbc0d2e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:22:56.254477 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-12bb9610-647d-47e5-ae06-048b314a15e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bf07e2d602d4323a160a446ddbc0d2e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:22:56.271399 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1466/2911] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:22:56 2022] DELETE /identity/v3/users/8bf07e2d602d4323a160a446ddbc0d2e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:22:56.275860 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8917b60-ec40-48ac-b096-fdf0a59b405b None None] REQUEST_METHOD: `POST` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:56.276362 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8917b60-ec40-48ac-b096-fdf0a59b405b None None] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:56.276475 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8917b60-ec40-48ac-b096-fdf0a59b405b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:56.295616 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.auth.core [None req-a8917b60-ec40-48ac-b096-fdf0a59b405b None None] MFA Rules not processed for user `e46b19c26ed14ce08997d6b7fd3a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81596) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:22:56.296804 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-a8917b60-ec40-48ac-b096-fdf0a59b405b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:22:56.354697 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1446/2912] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:22:56.364327 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb9610-647d-47e5-ae06-048b314a15e5 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:22:56.365692 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-12bb9610-647d-47e5-ae06-048b314a15e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:22:56.410358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:22:56.422752 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb9610-647d-47e5-ae06-048b314a15e5 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:22:56.426300 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b69d3b-24c3-443c-860c-e919bd2d762a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:22:56.426924 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b69d3b-24c3-443c-860c-e919bd2d762a service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:56.427202 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b69d3b-24c3-443c-860c-e919bd2d762a service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:56.427468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b69d3b-24c3-443c-860c-e919bd2d762a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:56.436429 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b69d3b-24c3-443c-860c-e919bd2d762a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:22:56.438347 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b69d3b-24c3-443c-860c-e919bd2d762a service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:22:56.447435 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1467/2913] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:22:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:22:56.705430 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8917b60-ec40-48ac-b096-fdf0a59b405b None None] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:22:56.706903 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-a8917b60-ec40-48ac-b096-fdf0a59b405b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:22:56.718980 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8917b60-ec40-48ac-b096-fdf0a59b405b None None] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:22:56.723717 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9317a9c-4997-4e12-b2d1-e6e3e5b4239c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:22:56.724368 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9317a9c-4997-4e12-b2d1-e6e3e5b4239c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:22:56.724654 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9317a9c-4997-4e12-b2d1-e6e3e5b4239c admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:22:56.724922 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9317a9c-4997-4e12-b2d1-e6e3e5b4239c admin admin] PATH_INFO: `/v3/projects/cb3e6765ec6743efa823b1c814cd5daf` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:22:56.726060 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9317a9c-4997-4e12-b2d1-e6e3e5b4239c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb3e6765ec6743efa823b1c814cd5daf)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:22:56.728077 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9317a9c-4997-4e12-b2d1-e6e3e5b4239c admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:22:56.774899 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-d9317a9c-4997-4e12-b2d1-e6e3e5b4239c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb3e6765ec6743efa823b1c814cd5daf', 'request_id': 'req-d9317a9c-4997-4e12-b2d1-e6e3e5b4239c'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:22:56.780437 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-d9317a9c-4997-4e12-b2d1-e6e3e5b4239c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb3e6765ec6743efa823b1c814cd5daf', 'request_id': 'req-d9317a9c-4997-4e12-b2d1-e6e3e5b4239c'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:22:56.784101 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:22:56.788918 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-d9317a9c-4997-4e12-b2d1-e6e3e5b4239c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb3e6765ec6743efa823b1c814cd5daf', 'request_id': 'req-d9317a9c-4997-4e12-b2d1-e6e3e5b4239c'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:22:56.804265 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-d9317a9c-4997-4e12-b2d1-e6e3e5b4239c admin admin] The token cache is being invalidate because project cb3e6765ec6743efa823b1c814cd5daf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:22:56.804567 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-d9317a9c-4997-4e12-b2d1-e6e3e5b4239c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:22:56.805280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-d9317a9c-4997-4e12-b2d1-e6e3e5b4239c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:22:56.806831 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1447/2914] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:22:56 2022] DELETE /identity/v3/projects/cb3e6765ec6743efa823b1c814cd5daf => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:23:06.443461 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=7 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:23:06.801008 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=4 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:24:05.924609 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b69d3b-24c3-443c-860c-e919bd2d762a service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:24:05.935158 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b69d3b-24c3-443c-860c-e919bd2d762a service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:24:06.001564 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975519f6-2853-434b-808c-44ca98cd10b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:24:06.002251 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975519f6-2853-434b-808c-44ca98cd10b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:24:06.002541 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975519f6-2853-434b-808c-44ca98cd10b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:24:06.002813 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975519f6-2853-434b-808c-44ca98cd10b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:24:06.004606 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-975519f6-2853-434b-808c-44ca98cd10b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:24:06.014310 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975519f6-2853-434b-808c-44ca98cd10b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:24:06.015763 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975519f6-2853-434b-808c-44ca98cd10b6 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:24:06.024344 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1468/2915] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:24:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:24:06.059427 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9317a9c-4997-4e12-b2d1-e6e3e5b4239c admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:24:06.061219 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-d9317a9c-4997-4e12-b2d1-e6e3e5b4239c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:24:06.075602 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9317a9c-4997-4e12-b2d1-e6e3e5b4239c admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:24:06.079325 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd24715b-2190-4341-a729-48b95f306a7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:24:06.079955 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd24715b-2190-4341-a729-48b95f306a7d service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:24:06.080228 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd24715b-2190-4341-a729-48b95f306a7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:24:06.080496 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd24715b-2190-4341-a729-48b95f306a7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:24:06.082211 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-dd24715b-2190-4341-a729-48b95f306a7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:24:06.153052 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd24715b-2190-4341-a729-48b95f306a7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:24:06.154484 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd24715b-2190-4341-a729-48b95f306a7d service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:24:06.166468 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1448/2916] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:24:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:24:07.348230 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975519f6-2853-434b-808c-44ca98cd10b6 service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:24:07.357033 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975519f6-2853-434b-808c-44ca98cd10b6 service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:24:07.361105 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe1a8e-5e0a-411a-8e35-62909c7140d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:24:07.361706 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe1a8e-5e0a-411a-8e35-62909c7140d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:24:07.361986 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe1a8e-5e0a-411a-8e35-62909c7140d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:24:07.362258 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe1a8e-5e0a-411a-8e35-62909c7140d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:24:07.364027 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-fbbe1a8e-5e0a-411a-8e35-62909c7140d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:24:07.378259 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbe1a8e-5e0a-411a-8e35-62909c7140d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:24:07.379695 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbe1a8e-5e0a-411a-8e35-62909c7140d8 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:24:07.390429 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1469/2917] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:24:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:24:08.243826 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd24715b-2190-4341-a729-48b95f306a7d service neutron] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:24:08.245330 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-dd24715b-2190-4341-a729-48b95f306a7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:24:08.259867 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd24715b-2190-4341-a729-48b95f306a7d service neutron] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:24:08.265967 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efa012-0feb-4f90-9d0f-4e243e6982dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:24:08.266574 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efa012-0feb-4f90-9d0f-4e243e6982dd service nova] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:24:08.267526 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efa012-0feb-4f90-9d0f-4e243e6982dd service nova] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:24:08.267807 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efa012-0feb-4f90-9d0f-4e243e6982dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:24:08.269769 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-f0efa012-0feb-4f90-9d0f-4e243e6982dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:24:08.283220 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0efa012-0feb-4f90-9d0f-4e243e6982dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:24:08.284774 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0efa012-0feb-4f90-9d0f-4e243e6982dd service nova] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:24:08.295631 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1449/2918] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:24:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:24:09.914426 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe1a8e-5e0a-411a-8e35-62909c7140d8 service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:24:09.923654 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe1a8e-5e0a-411a-8e35-62909c7140d8 service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:24:09.927260 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d92fd4-c2af-4615-9cb0-654478d14321 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:24:09.927784 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d92fd4-c2af-4615-9cb0-654478d14321 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:24:09.928059 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d92fd4-c2af-4615-9cb0-654478d14321 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:24:09.928326 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d92fd4-c2af-4615-9cb0-654478d14321 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:24:09.929988 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b5d92fd4-c2af-4615-9cb0-654478d14321 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:24:09.992212 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d92fd4-c2af-4615-9cb0-654478d14321 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:24:09.993611 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d92fd4-c2af-4615-9cb0-654478d14321 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:24:10.003107 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1470/2919] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:24:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:24:13.021165 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efa012-0feb-4f90-9d0f-4e243e6982dd service nova] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:24:13.023172 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-f0efa012-0feb-4f90-9d0f-4e243e6982dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:24:13.143665 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efa012-0feb-4f90-9d0f-4e243e6982dd service nova] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:24:13.148761 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c04091-7c50-4c89-97f3-3fb102ca41c9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce508bb9aa164a7fb1831a9b5ff5b932', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:24:13.149381 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c04091-7c50-4c89-97f3-3fb102ca41c9 service placement] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:24:13.149776 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c04091-7c50-4c89-97f3-3fb102ca41c9 service placement] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:24:13.150098 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c04091-7c50-4c89-97f3-3fb102ca41c9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:24:13.151765 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-52c04091-7c50-4c89-97f3-3fb102ca41c9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:24:13.164743 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c04091-7c50-4c89-97f3-3fb102ca41c9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:24:13.166438 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c04091-7c50-4c89-97f3-3fb102ca41c9 service placement] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:24:13.181427 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1450/2920] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:24:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:24:16.151740 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d92fd4-c2af-4615-9cb0-654478d14321 service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:24:16.153206 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b5d92fd4-c2af-4615-9cb0-654478d14321 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:24:16.167351 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d92fd4-c2af-4615-9cb0-654478d14321 service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:24:16.172092 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1604a5-1529-4be2-98fe-eca8c5edabfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:24:16.172660 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1604a5-1529-4be2-98fe-eca8c5edabfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:24:16.172985 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1604a5-1529-4be2-98fe-eca8c5edabfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:24:16.173255 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1604a5-1529-4be2-98fe-eca8c5edabfd admin admin] PATH_INFO: `/v3/users/0956fadcbd1e4bcd853eaf7cd5115d93` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:24:16.174438 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1604a5-1529-4be2-98fe-eca8c5edabfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0956fadcbd1e4bcd853eaf7cd5115d93)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:24:16.176365 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1604a5-1529-4be2-98fe-eca8c5edabfd admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:24:16.223829 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2f1604a5-1529-4be2-98fe-eca8c5edabfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0956fadcbd1e4bcd853eaf7cd5115d93'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:24:16.230608 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2f1604a5-1529-4be2-98fe-eca8c5edabfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0956fadcbd1e4bcd853eaf7cd5115d93'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:24:16.231240 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2f1604a5-1529-4be2-98fe-eca8c5edabfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0956fadcbd1e4bcd853eaf7cd5115d93'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:24:16.231840 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2f1604a5-1529-4be2-98fe-eca8c5edabfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0956fadcbd1e4bcd853eaf7cd5115d93'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:24:16.238627 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2f1604a5-1529-4be2-98fe-eca8c5edabfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0956fadcbd1e4bcd853eaf7cd5115d93'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:24:16.249168 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:24:16.256413 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1471/2921] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:24:16 2022] DELETE /identity/v3/users/0956fadcbd1e4bcd853eaf7cd5115d93 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 14:24:16.424560 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c04091-7c50-4c89-97f3-3fb102ca41c9 service placement] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:24:16.437093 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c04091-7c50-4c89-97f3-3fb102ca41c9 service placement] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:24:16.493833 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff51442-bb24-4aef-9543-6455b0cfe814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:24:16.494461 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff51442-bb24-4aef-9543-6455b0cfe814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:24:16.494738 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff51442-bb24-4aef-9543-6455b0cfe814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:24:16.495047 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff51442-bb24-4aef-9543-6455b0cfe814 admin admin] PATH_INFO: `/v3/projects/fd4a527e5cc441f5901536b58d4fe400` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:24:16.496158 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff51442-bb24-4aef-9543-6455b0cfe814 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd4a527e5cc441f5901536b58d4fe400)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:24:16.498070 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff51442-bb24-4aef-9543-6455b0cfe814 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:24:16.500267 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=100 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:24:16.544075 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-7ff51442-bb24-4aef-9543-6455b0cfe814 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd4a527e5cc441f5901536b58d4fe400', 'request_id': 'req-7ff51442-bb24-4aef-9543-6455b0cfe814'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:24:16.549604 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-7ff51442-bb24-4aef-9543-6455b0cfe814 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd4a527e5cc441f5901536b58d4fe400', 'request_id': 'req-7ff51442-bb24-4aef-9543-6455b0cfe814'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:24:16.556630 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-7ff51442-bb24-4aef-9543-6455b0cfe814 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd4a527e5cc441f5901536b58d4fe400', 'request_id': 'req-7ff51442-bb24-4aef-9543-6455b0cfe814'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:24:16.568377 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-7ff51442-bb24-4aef-9543-6455b0cfe814 admin admin] The token cache is being invalidate because project fd4a527e5cc441f5901536b58d4fe400 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:24:16.568663 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-7ff51442-bb24-4aef-9543-6455b0cfe814 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:24:16.569299 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-7ff51442-bb24-4aef-9543-6455b0cfe814 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:24:16.570765 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1451/2922] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:24:16 2022] DELETE /identity/v3/projects/fd4a527e5cc441f5901536b58d4fe400 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:24:16.575010 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1604a5-1529-4be2-98fe-eca8c5edabfd admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:24:16.576533 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-2f1604a5-1529-4be2-98fe-eca8c5edabfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:24:16.589956 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1604a5-1529-4be2-98fe-eca8c5edabfd admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:24:16.644204 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa66c669-f6f5-434e-8be2-c643e7f07054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:24:16.644821 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66c669-f6f5-434e-8be2-c643e7f07054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:24:16.645100 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66c669-f6f5-434e-8be2-c643e7f07054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:24:16.645370 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66c669-f6f5-434e-8be2-c643e7f07054 admin admin] PATH_INFO: `/v3/users/623f696b52574522a1fcab40de0f4764` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:24:16.646477 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa66c669-f6f5-434e-8be2-c643e7f07054 admin admin] RBAC: Authorizing `identity:delete_user(user_id=623f696b52574522a1fcab40de0f4764)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:24:16.648400 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa66c669-f6f5-434e-8be2-c643e7f07054 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:24:16.697750 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-fa66c669-f6f5-434e-8be2-c643e7f07054 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '623f696b52574522a1fcab40de0f4764'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:24:16.705692 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-fa66c669-f6f5-434e-8be2-c643e7f07054 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '623f696b52574522a1fcab40de0f4764'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:24:16.706313 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-fa66c669-f6f5-434e-8be2-c643e7f07054 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '623f696b52574522a1fcab40de0f4764'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:24:16.706940 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-fa66c669-f6f5-434e-8be2-c643e7f07054 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '623f696b52574522a1fcab40de0f4764'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:24:16.713974 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-fa66c669-f6f5-434e-8be2-c643e7f07054 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '623f696b52574522a1fcab40de0f4764'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:24:16.730468 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1472/2923] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:24:16 2022] DELETE /identity/v3/users/623f696b52574522a1fcab40de0f4764 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:24:16.910060 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff51442-bb24-4aef-9543-6455b0cfe814 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:24:16.911574 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-7ff51442-bb24-4aef-9543-6455b0cfe814 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:24:16.925164 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff51442-bb24-4aef-9543-6455b0cfe814 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:24:16.982373 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbce8f9-e478-4e02-8843-a6257b524985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:24:16.983052 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbce8f9-e478-4e02-8843-a6257b524985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:24:16.983315 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbce8f9-e478-4e02-8843-a6257b524985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:24:16.983586 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbce8f9-e478-4e02-8843-a6257b524985 admin admin] PATH_INFO: `/v3/projects/f35e8d128cd44441b85c0ab5d5fe14cf` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:24:16.984691 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbce8f9-e478-4e02-8843-a6257b524985 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f35e8d128cd44441b85c0ab5d5fe14cf)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:24:16.987260 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbce8f9-e478-4e02-8843-a6257b524985 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:24:17.033085 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-0fbce8f9-e478-4e02-8843-a6257b524985 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f35e8d128cd44441b85c0ab5d5fe14cf', 'request_id': 'req-0fbce8f9-e478-4e02-8843-a6257b524985'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:24:17.038657 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-0fbce8f9-e478-4e02-8843-a6257b524985 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f35e8d128cd44441b85c0ab5d5fe14cf', 'request_id': 'req-0fbce8f9-e478-4e02-8843-a6257b524985'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:24:17.045776 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-0fbce8f9-e478-4e02-8843-a6257b524985 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f35e8d128cd44441b85c0ab5d5fe14cf', 'request_id': 'req-0fbce8f9-e478-4e02-8843-a6257b524985'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:24:17.057305 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-0fbce8f9-e478-4e02-8843-a6257b524985 admin admin] The token cache is being invalidate because project f35e8d128cd44441b85c0ab5d5fe14cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:24:17.057588 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-0fbce8f9-e478-4e02-8843-a6257b524985 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:24:17.058242 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-0fbce8f9-e478-4e02-8843-a6257b524985 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:24:17.060004 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1452/2924] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:24:16 2022] DELETE /identity/v3/projects/f35e8d128cd44441b85c0ab5d5fe14cf => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:24:26.727065 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=55,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:24:27.054341 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=74,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:24:52.047057 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654f34a-d964-4d7f-9004-a288f8109cf8 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:24:52.047635 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654f34a-d964-4d7f-9004-a288f8109cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:24:52.047783 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654f34a-d964-4d7f-9004-a288f8109cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:24:52.067525 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-3654f34a-d964-4d7f-9004-a288f8109cf8 None None] MFA Rules not processed for user `c31fb669f8f14c8bbd5b7ec3450c930d`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:24:52.068727 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-3654f34a-d964-4d7f-9004-a288f8109cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:24:52.126150 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1473/2925] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:24:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:24:52.880564 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbce8f9-e478-4e02-8843-a6257b524985 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:24:52.882056 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-0fbce8f9-e478-4e02-8843-a6257b524985 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:24:52.951423 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbce8f9-e478-4e02-8843-a6257b524985 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:24:52.956505 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06032c10-a2e8-4899-9479-d8edc4827bf0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce508bb9aa164a7fb1831a9b5ff5b932', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:24:52.957106 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06032c10-a2e8-4899-9479-d8edc4827bf0 service placement] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:24:52.957385 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06032c10-a2e8-4899-9479-d8edc4827bf0 service placement] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:24:52.957654 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06032c10-a2e8-4899-9479-d8edc4827bf0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:24:52.959384 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-06032c10-a2e8-4899-9479-d8edc4827bf0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:24:52.970562 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06032c10-a2e8-4899-9479-d8edc4827bf0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:24:52.972163 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06032c10-a2e8-4899-9479-d8edc4827bf0 service placement] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:24:52.981839 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1453/2926] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:24:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:25:02.119433 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=25 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:25:02.977721 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=32 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:25:06.605476 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654f34a-d964-4d7f-9004-a288f8109cf8 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:25:06.606987 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-3654f34a-d964-4d7f-9004-a288f8109cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:25:06.697567 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654f34a-d964-4d7f-9004-a288f8109cf8 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:25:06.703259 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00bdf56-63b3-4917-9935-3b44e008d234 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:25:06.703803 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00bdf56-63b3-4917-9935-3b44e008d234 service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:25:06.704079 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00bdf56-63b3-4917-9935-3b44e008d234 service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:25:06.704349 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00bdf56-63b3-4917-9935-3b44e008d234 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:25:06.706170 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b00bdf56-63b3-4917-9935-3b44e008d234 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:25:06.718546 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00bdf56-63b3-4917-9935-3b44e008d234 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:25:06.720140 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00bdf56-63b3-4917-9935-3b44e008d234 service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:25:06.732024 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1474/2927] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:25:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:25:16.727027 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=34 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:26:27.111704 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06032c10-a2e8-4899-9479-d8edc4827bf0 service placement] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:26:27.113094 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-06032c10-a2e8-4899-9479-d8edc4827bf0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:26:27.125853 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06032c10-a2e8-4899-9479-d8edc4827bf0 service placement] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:26:27.130739 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b09ec-12e4-4f0d-a4f7-830ab466ba5b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce508bb9aa164a7fb1831a9b5ff5b932', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:26:27.131356 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b09ec-12e4-4f0d-a4f7-830ab466ba5b service placement] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:26:27.131634 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b09ec-12e4-4f0d-a4f7-830ab466ba5b service placement] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:26:27.131905 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b09ec-12e4-4f0d-a4f7-830ab466ba5b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:26:27.133548 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-9b3b09ec-12e4-4f0d-a4f7-830ab466ba5b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:26:27.145146 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3b09ec-12e4-4f0d-a4f7-830ab466ba5b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:26:27.146713 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3b09ec-12e4-4f0d-a4f7-830ab466ba5b service placement] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:26:27.157068 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1454/2928] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:26:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:26:37.152370 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=6 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:26:45.111538 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753dcde0-c415-483a-8f11-e03e2f6ea5ea None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:26:45.112060 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753dcde0-c415-483a-8f11-e03e2f6ea5ea None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:26:45.112060 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753dcde0-c415-483a-8f11-e03e2f6ea5ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:26:45.131416 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-753dcde0-c415-483a-8f11-e03e2f6ea5ea None None] MFA Rules not processed for user `c31fb669f8f14c8bbd5b7ec3450c930d`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 14:26:45.132605 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-753dcde0-c415-483a-8f11-e03e2f6ea5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:26:45.140098 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1475/2929] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 12 14:26:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 14:26:45.455279 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b09ec-12e4-4f0d-a4f7-830ab466ba5b service placement] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:26:45.456689 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-9b3b09ec-12e4-4f0d-a4f7-830ab466ba5b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:26:45.512450 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b09ec-12e4-4f0d-a4f7-830ab466ba5b service placement] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:26:45.516471 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ade4d8-4bbf-4e75-a0b8-93a26b170bd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:26:45.517073 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ade4d8-4bbf-4e75-a0b8-93a26b170bd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:26:45.517346 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ade4d8-4bbf-4e75-a0b8-93a26b170bd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:26:45.517613 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ade4d8-4bbf-4e75-a0b8-93a26b170bd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:26:45.519331 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-a1ade4d8-4bbf-4e75-a0b8-93a26b170bd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:26:45.528980 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ade4d8-4bbf-4e75-a0b8-93a26b170bd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:26:45.530472 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ade4d8-4bbf-4e75-a0b8-93a26b170bd0 service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:26:45.538431 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1455/2930] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:26:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:26:55.125947 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:26:55.534897 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=26 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:27:00.537236 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753dcde0-c415-483a-8f11-e03e2f6ea5ea None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:27:00.552272 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753dcde0-c415-483a-8f11-e03e2f6ea5ea None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:27:00.558018 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c7cff-66fe-499b-b1a7-16579cdd369b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '854ab2dcec5a48b18f409b33cb61b5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:27:00.558624 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c7cff-66fe-499b-b1a7-16579cdd369b service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:27:00.558927 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c7cff-66fe-499b-b1a7-16579cdd369b service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:27:00.559199 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c7cff-66fe-499b-b1a7-16579cdd369b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:27:00.561028 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-fa4c7cff-66fe-499b-b1a7-16579cdd369b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:27:00.573214 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c7cff-66fe-499b-b1a7-16579cdd369b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:27:00.574779 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c7cff-66fe-499b-b1a7-16579cdd369b service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:27:00.588592 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1476/2931] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:27:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:27:10.583948 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=6 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:28:28.826312 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ade4d8-4bbf-4e75-a0b8-93a26b170bd0 service neutron] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:28:28.827747 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-a1ade4d8-4bbf-4e75-a0b8-93a26b170bd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:28:28.838606 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ade4d8-4bbf-4e75-a0b8-93a26b170bd0 service neutron] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:28:28.844486 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2cd2ab-876a-4b60-a7f1-b224a72f8eca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:28:28.845117 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2cd2ab-876a-4b60-a7f1-b224a72f8eca service neutron] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:28:28.845393 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2cd2ab-876a-4b60-a7f1-b224a72f8eca service neutron] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:28:28.845661 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2cd2ab-876a-4b60-a7f1-b224a72f8eca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:28:28.847349 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-8f2cd2ab-876a-4b60-a7f1-b224a72f8eca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:28:28.857930 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2cd2ab-876a-4b60-a7f1-b224a72f8eca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:28:28.859369 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2cd2ab-876a-4b60-a7f1-b224a72f8eca service neutron] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:28:28.867914 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1456/2932] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:28:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:28:38.864083 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=6 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:28:42.190626 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c7cff-66fe-499b-b1a7-16579cdd369b service nova] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:28:42.200617 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c7cff-66fe-499b-b1a7-16579cdd369b service nova] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:28:42.204218 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a7fe2-910a-4edc-9dd3-2634eae27b29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '865b1647468c4efe8efe4febe2203a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:28:42.204824 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a7fe2-910a-4edc-9dd3-2634eae27b29 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:28:42.205101 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a7fe2-910a-4edc-9dd3-2634eae27b29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:28:42.205370 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a7fe2-910a-4edc-9dd3-2634eae27b29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:28:42.207097 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-2f9a7fe2-910a-4edc-9dd3-2634eae27b29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:28:42.268432 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a7fe2-910a-4edc-9dd3-2634eae27b29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:28:42.269830 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a7fe2-910a-4edc-9dd3-2634eae27b29 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:28:42.279728 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1477/2933] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:28:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:28:48.092049 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2cd2ab-876a-4b60-a7f1-b224a72f8eca service neutron] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:28:48.105373 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2cd2ab-876a-4b60-a7f1-b224a72f8eca service neutron] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:28:48.110808 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6c302-69cb-46ce-93d7-2fce97f98981 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce508bb9aa164a7fb1831a9b5ff5b932', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:28:48.111651 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6c302-69cb-46ce-93d7-2fce97f98981 service placement] REQUEST_METHOD: `GET` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:28:48.112113 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6c302-69cb-46ce-93d7-2fce97f98981 service placement] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:28:48.112547 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6c302-69cb-46ce-93d7-2fce97f98981 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:28:48.118280 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-caa6c302-69cb-46ce-93d7-2fce97f98981 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:28:48.131233 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa6c302-69cb-46ce-93d7-2fce97f98981 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:28:48.132826 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa6c302-69cb-46ce-93d7-2fce97f98981 service placement] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:28:48.144199 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1457/2934] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:28:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:28:48.159005 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a7fe2-910a-4edc-9dd3-2634eae27b29 service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:28:48.160421 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-2f9a7fe2-910a-4edc-9dd3-2634eae27b29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:28:48.174219 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a7fe2-910a-4edc-9dd3-2634eae27b29 service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:28:48.179119 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef5c8b-4846-4f58-a64a-dd0921171ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:28:48.179781 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef5c8b-4846-4f58-a64a-dd0921171ff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:28:48.180063 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef5c8b-4846-4f58-a64a-dd0921171ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:28:48.180338 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef5c8b-4846-4f58-a64a-dd0921171ff6 admin admin] PATH_INFO: `/v3/users/c31fb669f8f14c8bbd5b7ec3450c930d` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:28:48.181754 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ef5c8b-4846-4f58-a64a-dd0921171ff6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c31fb669f8f14c8bbd5b7ec3450c930d)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:28:48.184387 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ef5c8b-4846-4f58-a64a-dd0921171ff6 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:28:48.237652 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-88ef5c8b-4846-4f58-a64a-dd0921171ff6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c31fb669f8f14c8bbd5b7ec3450c930d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:28:48.247057 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-88ef5c8b-4846-4f58-a64a-dd0921171ff6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c31fb669f8f14c8bbd5b7ec3450c930d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:28:48.247785 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-88ef5c8b-4846-4f58-a64a-dd0921171ff6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c31fb669f8f14c8bbd5b7ec3450c930d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:28:48.248393 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-88ef5c8b-4846-4f58-a64a-dd0921171ff6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c31fb669f8f14c8bbd5b7ec3450c930d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:28:48.255970 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-88ef5c8b-4846-4f58-a64a-dd0921171ff6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c31fb669f8f14c8bbd5b7ec3450c930d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:28:48.274011 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1478/2935] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:28:48 2022] DELETE /identity/v3/users/c31fb669f8f14c8bbd5b7ec3450c930d => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:28:48.441820 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6c302-69cb-46ce-93d7-2fce97f98981 service placement] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:28:48.455755 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6c302-69cb-46ce-93d7-2fce97f98981 service placement] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:28:48.513863 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b728732-fc20-423d-9724-99a67a181465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:28:48.514513 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b728732-fc20-423d-9724-99a67a181465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:28:48.514793 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b728732-fc20-423d-9724-99a67a181465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:28:48.515093 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b728732-fc20-423d-9724-99a67a181465 admin admin] PATH_INFO: `/v3/projects/caed5793408e49edb40acad5410e02a9` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:28:48.516204 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b728732-fc20-423d-9724-99a67a181465 admin admin] RBAC: Authorizing `identity:delete_project(project_id=caed5793408e49edb40acad5410e02a9)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:28:48.518073 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b728732-fc20-423d-9724-99a67a181465 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:28:48.563936 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-8b728732-fc20-423d-9724-99a67a181465 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'caed5793408e49edb40acad5410e02a9', 'request_id': 'req-8b728732-fc20-423d-9724-99a67a181465'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:28:48.569435 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-8b728732-fc20-423d-9724-99a67a181465 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'caed5793408e49edb40acad5410e02a9', 'request_id': 'req-8b728732-fc20-423d-9724-99a67a181465'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:28:48.576405 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-8b728732-fc20-423d-9724-99a67a181465 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'caed5793408e49edb40acad5410e02a9', 'request_id': 'req-8b728732-fc20-423d-9724-99a67a181465'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:28:48.587180 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-8b728732-fc20-423d-9724-99a67a181465 admin admin] The token cache is being invalidate because project caed5793408e49edb40acad5410e02a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:28:48.587462 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-8b728732-fc20-423d-9724-99a67a181465 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:28:48.588083 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-8b728732-fc20-423d-9724-99a67a181465 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:28:48.590048 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1458/2936] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:28:48 2022] DELETE /identity/v3/projects/caed5793408e49edb40acad5410e02a9 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:28:48.593997 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef5c8b-4846-4f58-a64a-dd0921171ff6 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:28:48.595451 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-88ef5c8b-4846-4f58-a64a-dd0921171ff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:28:48.608735 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef5c8b-4846-4f58-a64a-dd0921171ff6 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:28:48.663469 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d43be-12cb-49a4-8abc-d5ec16315022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:28:48.664099 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d43be-12cb-49a4-8abc-d5ec16315022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:28:48.664378 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d43be-12cb-49a4-8abc-d5ec16315022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:28:48.664648 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d43be-12cb-49a4-8abc-d5ec16315022 admin admin] PATH_INFO: `/v3/users/a9a497f1250e4fc681892928a7479fc2` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:28:48.665777 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d43be-12cb-49a4-8abc-d5ec16315022 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9a497f1250e4fc681892928a7479fc2)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:28:48.668251 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d43be-12cb-49a4-8abc-d5ec16315022 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:28:48.674494 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:28:48.716490 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-810d43be-12cb-49a4-8abc-d5ec16315022 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9a497f1250e4fc681892928a7479fc2'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:28:48.724707 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-810d43be-12cb-49a4-8abc-d5ec16315022 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9a497f1250e4fc681892928a7479fc2'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:28:48.725313 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-810d43be-12cb-49a4-8abc-d5ec16315022 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9a497f1250e4fc681892928a7479fc2'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:28:48.725907 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-810d43be-12cb-49a4-8abc-d5ec16315022 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9a497f1250e4fc681892928a7479fc2'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:28:48.733582 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-810d43be-12cb-49a4-8abc-d5ec16315022 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9a497f1250e4fc681892928a7479fc2'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:28:48.750358 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1479/2937] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 12 14:28:48 2022] DELETE /identity/v3/users/a9a497f1250e4fc681892928a7479fc2 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:28:48.920902 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b728732-fc20-423d-9724-99a67a181465 admin admin] Authenticating user token {{(pid=81596) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:28:48.922338 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.fernet_utils [None req-8b728732-fc20-423d-9724-99a67a181465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81596) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:28:48.935554 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b728732-fc20-423d-9724-99a67a181465 admin admin] Validating token access rules against request {{(pid=81596) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:28:49.000924 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d537f-c071-4849-b092-94e8aa848705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46b19c26ed14ce08997d6b7fd3a2efb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b159fcc4f05d45de88c679e235640813', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81596) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:28:49.001637 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d537f-c071-4849-b092-94e8aa848705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:28:49.001812 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d537f-c071-4849-b092-94e8aa848705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:28:49.002088 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d537f-c071-4849-b092-94e8aa848705 admin admin] PATH_INFO: `/v3/projects/438b10e90ef242a9a61e81d427823e44` {{(pid=81596) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:28:49.003241 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d537f-c071-4849-b092-94e8aa848705 admin admin] RBAC: Authorizing `identity:delete_project(project_id=438b10e90ef242a9a61e81d427823e44)` {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:28:49.005134 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d537f-c071-4849-b092-94e8aa848705 admin admin] RBAC: Authorization granted {{(pid=81596) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:28:49.026594 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:28:49.054293 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-4c4d537f-c071-4849-b092-94e8aa848705 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '438b10e90ef242a9a61e81d427823e44', 'request_id': 'req-4c4d537f-c071-4849-b092-94e8aa848705'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:28:49.060055 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-4c4d537f-c071-4849-b092-94e8aa848705 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '438b10e90ef242a9a61e81d427823e44', 'request_id': 'req-4c4d537f-c071-4849-b092-94e8aa848705'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:28:49.067193 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-4c4d537f-c071-4849-b092-94e8aa848705 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '438b10e90ef242a9a61e81d427823e44', 'request_id': 'req-4c4d537f-c071-4849-b092-94e8aa848705'} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:28:49.080270 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-4c4d537f-c071-4849-b092-94e8aa848705 admin admin] The token cache is being invalidate because project 438b10e90ef242a9a61e81d427823e44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81596) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 14:28:49.080626 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-4c4d537f-c071-4849-b092-94e8aa848705 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:28:49.081302 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG keystone.notifications [None req-4c4d537f-c071-4849-b092-94e8aa848705 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81596) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 14:28:49.083165 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: [pid: 81596|app: 0|req: 1459/2938] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 12 14:28:48 2022] DELETE /identity/v3/projects/438b10e90ef242a9a61e81d427823e44 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 14:28:58.746915 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:28:59.076621 dv-cinder-820027-25-iscsi devstack@keystone.service[81596]: DEBUG dbcounter [-] [81596] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 14:31:32.079930 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d43be-12cb-49a4-8abc-d5ec16315022 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 14:31:32.081310 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-810d43be-12cb-49a4-8abc-d5ec16315022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:31:32.217022 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d43be-12cb-49a4-8abc-d5ec16315022 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 14:31:32.277169 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59852412-95cf-4abc-b9c9-86fbaba2b667 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce508bb9aa164a7fb1831a9b5ff5b932', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f666a617f8d14eb18bbeb076032b82f2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 14:31:32.278196 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59852412-95cf-4abc-b9c9-86fbaba2b667 service placement] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 14:31:32.278605 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59852412-95cf-4abc-b9c9-86fbaba2b667 service placement] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 14:31:32.278903 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59852412-95cf-4abc-b9c9-86fbaba2b667 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 14:31:32.280682 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-59852412-95cf-4abc-b9c9-86fbaba2b667 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 14:31:32.294083 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59852412-95cf-4abc-b9c9-86fbaba2b667 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 14:31:32.295696 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59852412-95cf-4abc-b9c9-86fbaba2b667 service placement] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 14:31:32.306464 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1480/2939] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 12 14:31:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 14:31:42.301403 dv-cinder-820027-25-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}